From Sagir at mellanox.co.il Tue Jul 1 00:06:26 2008 From: Sagir at mellanox.co.il (Sagi Rotem) Date: Tue, 1 Jul 2008 10:06:26 +0300 Subject: [ofa-general] Re: performance drop for datagram mode with the newconnectx FW In-Reply-To: Message-ID: <40FA0A8088E8A441973D37502F00933E309E4E@mtlexch01.mtl.com> Or, Are these results after you have applied Eli's fix ? Sagi -----Original Message----- From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Or Gerlitz Sent: Tuesday, July 01, 2008 9:42 AM To: Oren Meron Cc: Eli Cohen; general at lists.openfabrics.org Subject: [ofa-general] Re: performance drop for datagram mode with the newconnectx FW > Using the new connectx FW (2.5), I see performance drop to almost zero > with ipoib datagram mode. The code that runs on these systems is ofed > 1.3 and not mainline kernel, details below. > > Running netperf With connected mode (64k MTU) I get about 950MB/s > where with datagram mode (2k MTU) I get only 20-40MB/s. I used to see > about 650MB/s and above with FW 2.3 and datagram mode. Not that it > could explain the drop, but the NIC reports to the OS stateless > offload support - /sys/class/net/ib1/features is 0x11423 I think to manage narrowing this a little further, the issue seems most notable under long messages sent over datagram mode, could be an issue in the LSO engine ?! see in this table, ==================================================== mode mtu size bw note ==================================================== datagram 2044 64000 30 <---------- problem datagram 2044 2000 430 datagram 2044 2000 300 TCP_NODELAY ----------------------------------------------------- connected 2044 64000 450 connected 2044 2000 450 connected 2044 2000 300 TCP_NODELAY --------------------------------------------------- connected 64000 64000 930 connected 64000 2000 930 connected 64000 2000 470 TCP_NODELAY ==================================================== notes: - in all cases, I have set the mtu manually - verbs tests of bidirectional bandwidth shows that the HCA UD and RC engines work very well, and that the cables/switch etc are operating fine as the SDR limit is easily reached. # qperf -li mlx4_0:2 -ri mlx4_0:1 172.25.5.77 -m 2000 -t 10 rc_bi_bw rc_bi_bw: bw = 1.91 GB/sec # qperf -li mlx4_0:2 -ri mlx4_0:1 172.25.5.77 -m 2000 -t 10 ud_bi_bw ud_bi_bw: send_bw = 1.95 GB/sec recv_bw = 1.95 GB/sec datagram mode mtu 2044 # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 64000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 33.00 MBytes/s over 10.31 seconds Interim result: 22.27 MBytes/s over 1.48 seconds Interim result: 34.22 MBytes/s over 2.51 seconds Interim result: 34.05 MBytes/s over 1.01 seconds Interim result: 22.88 MBytes/s over 1.49 seconds Interim result: 30.03 MBytes/s over 1.00 seconds Interim result: 28.26 MBytes/s over 1.01 seconds Interim result: 28.21 MBytes/s over 1.00 seconds Interim result: 14.26 MBytes/s over 1.98 seconds # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 2000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 429.54 MBytes/s over 1.00 seconds Interim result: 429.22 MBytes/s over 1.00 seconds Interim result: 429.50 MBytes/s over 1.00 seconds Interim result: 429.47 MBytes/s over 1.00 seconds Interim result: 429.35 MBytes/s over 1.00 seconds Interim result: 429.10 MBytes/s over 1.00 seconds Interim result: 429.16 MBytes/s over 1.00 seconds # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 2000 -D L,R TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : nodelay : demo Interim result: 294.15 MBytes/s over 1.32 seconds Interim result: 294.27 MBytes/s over 1.00 seconds Interim result: 294.98 MBytes/s over 1.00 seconds Interim result: 294.74 MBytes/s over 1.00 seconds Interim result: 294.45 MBytes/s over 1.00 seconds Interim result: 294.53 MBytes/s over 1.00 seconds Interim result: 294.35 MBytes/s over 1.00 seconds Interim result: 294.43 MBytes/s over 1.00 seconds connected mode mtu 2044 # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 64000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 161.05 MBytes/s over 1.71 seconds Interim result: 231.38 MBytes/s over 1.00 seconds Interim result: 231.16 MBytes/s over 1.00 seconds Interim result: 232.40 MBytes/s over 1.00 seconds Interim result: 322.28 MBytes/s over 1.00 seconds Interim result: 448.25 MBytes/s over 1.00 seconds Interim result: 448.54 MBytes/s over 1.00 seconds Interim result: 448.36 MBytes/s over 1.00 seconds Interim result: 448.52 MBytes/s over 1.00 seconds Interim result: 448.18 MBytes/s over 1.00 seconds Interim result: 448.06 MBytes/s over 1.00 seconds Interim result: 448.26 MBytes/s over 1.00 seconds Interim result: 448.41 MBytes/s over 1.00 seconds # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 2000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 454.45 MBytes/s over 1.00 seconds Interim result: 455.10 MBytes/s over 1.00 seconds Interim result: 454.87 MBytes/s over 1.00 seconds Interim result: 454.64 MBytes/s over 1.00 seconds Interim result: 454.30 MBytes/s over 1.00 seconds Interim result: 454.11 MBytes/s over 1.00 seconds Interim result: 454.08 MBytes/s over 1.00 seconds Interim result: 454.07 MBytes/s over 1.00 seconds Interim result: 454.06 MBytes/s over 1.00 seconds Interim result: 454.19 MBytes/s over 1.00 seconds Interim result: 454.42 MBytes/s over 1.00 seconds Interim result: 454.31 MBytes/s over 1.00 seconds # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 2000 -D L,R TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : nodelay : demo Interim result: 305.93 MBytes/s over 1.00 seconds Interim result: 306.84 MBytes/s over 1.00 seconds Interim result: 306.88 MBytes/s over 1.00 seconds Interim result: 306.80 MBytes/s over 1.00 seconds Interim result: 306.69 MBytes/s over 1.00 seconds Interim result: 306.41 MBytes/s over 1.00 seconds Interim result: 307.25 MBytes/s over 1.00 seconds Interim result: 307.56 MBytes/s over 1.00 seconds Interim result: 307.51 MBytes/s over 1.00 seconds Interim result: 307.58 MBytes/s over 1.00 seconds connected mode mtu 64000 # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 64000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 932.87 MBytes/s over 1.21 seconds Interim result: 931.33 MBytes/s over 1.00 seconds Interim result: 930.20 MBytes/s over 1.00 seconds Interim result: 929.64 MBytes/s over 1.00 seconds Interim result: 931.16 MBytes/s over 1.00 seconds Interim result: 929.96 MBytes/s over 1.00 seconds # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 2000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 932.82 MBytes/s over 1.46 seconds Interim result: 930.69 MBytes/s over 1.00 seconds Interim result: 931.36 MBytes/s over 1.00 seconds Interim result: 930.05 MBytes/s over 1.00 seconds Interim result: 931.40 MBytes/s over 1.00 seconds Interim result: 930.07 MBytes/s over 1.00 seconds # netperf -H 10.10.0.90 -fM -l 600 -D 1, -- -m 2000 -D L,R TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : nodelay : demo Interim result: 475.01 MBytes/s over 1.58 seconds Interim result: 474.22 MBytes/s over 1.00 seconds Interim result: 474.25 MBytes/s over 1.00 seconds Interim result: 473.14 MBytes/s over 1.00 seconds Interim result: 473.62 MBytes/s over 1.00 seconds Interim result: 474.37 MBytes/s over 1.00 seconds Interim result: 473.81 MBytes/s over 1.00 seconds _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From ogerlitz at voltaire.com Tue Jul 1 00:08:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 01 Jul 2008 10:08:38 +0300 Subject: [ofa-general] Re: performance drop for datagram mode with the newconnectx FW In-Reply-To: <40FA0A8088E8A441973D37502F00933E309E4E@mtlexch01.mtl.com> References: <40FA0A8088E8A441973D37502F00933E309E4E@mtlexch01.mtl.com> Message-ID: <4869D7F6.8040809@voltaire.com> Sagi Rotem wrote: > Are these results after you have applied Eli's fix ? > As I wrote, in all cases I have applied --- manual mtu setting -- for which Eli's fix is not relevant, you can note that with 2k mtu I get quite good results for small packets and very bad for large packets. Or. From eli at dev.mellanox.co.il Tue Jul 1 00:15:59 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 1 Jul 2008 10:15:59 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <4869CE9C.2050408@voltaire.com> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> Message-ID: <20080701071559.GE6024@mtls03> On Tue, Jul 01, 2008 at 09:28:44AM +0300, Or Gerlitz wrote: > Please note that a flow which invokes call_netdevice_notifiers() should be > wrapped by rtnl_lock()/rtnl_unlock() and the context should be made > sleepable /non-atomic, etc the the consumers of this event might go to > sleep (eg on memory allocation for the skb of the netlink event to user > space etc). I am not calling call_netdevice_notifiers() directly but through dev_set_mtu() and the only other place I see that makes use of dev_set_mtu() is bond_change_mtu() and it does not use rtnl_lock()/rtnl_unlock(). > > Eli, yes, using dev_set_mtu() seems the correct way to go, even though I > don't think it explains any of the ipoib problems I have came across over > the last days, which I reported to you on the list, so in that respect I > don't really understand your other email where you have pointed to me the > mtu/setting issue. > I was able to see bad results when moving to CM mode which were resolved when I took that approach. Maybe we still have issues so let's take them one by one. Could you apply the patches I sent and check again? Also, please use the same approach at ipoib_multicast.c as Roland noted. I used this here and it does not have any bad impact, and I think it is appropriate. From Sagir at mellanox.co.il Tue Jul 1 00:17:11 2008 From: Sagir at mellanox.co.il (Sagi Rotem) Date: Tue, 1 Jul 2008 10:17:11 +0300 Subject: [ofa-general] Re: performance drop for datagram mode with the newconnectx FW In-Reply-To: <4869D7F6.8040809@voltaire.com> Message-ID: <40FA0A8088E8A441973D37502F00933E309E92@mtlexch01.mtl.com> See Eli's response in the other mail, lets keep to 1 thread on this subject. sagi -----Original Message----- From: Or Gerlitz [mailto:ogerlitz at voltaire.com] Sent: Tuesday, July 01, 2008 10:09 AM To: Sagi Rotem Cc: Oren Meron; Eli Cohen; general at lists.openfabrics.org Subject: Re: [ofa-general] Re: performance drop for datagram mode with the newconnectx FW Sagi Rotem wrote: > Are these results after you have applied Eli's fix ? > As I wrote, in all cases I have applied --- manual mtu setting -- for which Eli's fix is not relevant, you can note that with 2k mtu I get quite good results for small packets and very bad for large packets. Or. From eli at dev.mellanox.co.il Tue Jul 1 00:19:34 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 1 Jul 2008 10:19:34 +0300 Subject: [ofa-general] Re: performance drop for datagram mode with the new connectx FW In-Reply-To: References: Message-ID: <20080701071934.GF6024@mtls03> On Tue, Jul 01, 2008 at 09:41:30AM +0300, Or Gerlitz wrote: > notes: > > - in all cases, I have set the mtu manually Or, just to be sure the kernel does not filter out MTU changes that have the same value, you can either apply the patches or make sure that when you change the value, you go through another value. For example: ifconfig ib0 mtu 2000 ifconfig ib0 mtu 2044 From eli at dev.mellanox.co.il Tue Jul 1 00:45:36 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 1 Jul 2008 10:45:36 +0300 Subject: [ofa-general] Re: performance drop for datagram mode with the new connectx FW In-Reply-To: <20080701071934.GF6024@mtls03> References: <20080701071934.GF6024@mtls03> Message-ID: <20080701074536.GG6024@mtls03> On Tue, Jul 01, 2008 at 10:19:34AM +0300, Eli Cohen wrote: > On Tue, Jul 01, 2008 at 09:41:30AM +0300, Or Gerlitz wrote: > > notes: > > > > - in all cases, I have set the mtu manually > > Or, just to be sure the kernel does not filter out MTU changes that > have the same value, you can either apply the patches or make sure > that when you change the value, you go through another value. For > example: > > ifconfig ib0 mtu 2000 > ifconfig ib0 mtu 2044 > I was just able to verify that the above suggestion is required, at least on my systems (SLES 10). This is of course valid for changing to CM mode too. From vlad at mellanox.co.il Tue Jul 1 00:49:18 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 1 Jul 2008 10:49:18 +0300 Subject: [ofa-general] [PATCH] Fix %setup macro following RPM name change. Message-ID: <20080701074918.GA13105@mellanox.co.il> From: Vladimir Sokolovsky Signed-off-by: Vladimir Sokolovsky --- dapl.spec.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/dapl.spec.in b/dapl.spec.in index 89351c9..8282a97 100644 --- a/dapl.spec.in +++ b/dapl.spec.in @@ -72,7 +72,7 @@ Requires: %{name} = %{version}-%{release} Useful test suites to validate uDAPL library API's. %prep -%setup -q +%setup -q -n dapl-%{version} %build %configure -- 1.5.2.5 From ogerlitz at voltaire.com Tue Jul 1 01:04:53 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 01 Jul 2008 11:04:53 +0300 Subject: [ofa-general] Re: performance drop for datagram mode with the new connectx FW In-Reply-To: <20080701074536.GG6024@mtls03> References: <20080701071934.GF6024@mtls03> <20080701074536.GG6024@mtls03> Message-ID: <4869E525.8050809@voltaire.com> Eli Cohen wrote: > I was just able to verify that the above suggestion is required, at least on my systems (SLES 10). This is of course valid for changing to CM mode too. Eli, just to summarize, the above is fair-enough for the problem you saw, but I see something else: > mode mtu size bw note > ==================================================== > datagram 2044 64000 30 <---------- problem > datagram 2044 2000 430 > datagram 2044 2000 300 TCP_NODELAY > ----------------------------------------------------- on the same settings, the datagram mode bw changes from 30 MB/s to 430MB/s depending on the message size used by netperf, so its a different issue. Or. From tziporet at dev.mellanox.co.il Tue Jul 1 02:26:04 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 01 Jul 2008 12:26:04 +0300 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (June 30, 2008) In-Reply-To: <4868F1A3.1030708@opengridcomputing.com> References: <40FA0A8088E8A441973D37502F00933E2B498C@mtlexch01.mtl.com> <4868F1A3.1030708@opengridcomputing.com> Message-ID: <4869F82C.2000003@mellanox.co.il> Steve Wise wrote: > The MEM_MGT_EXTENSIONS patches are in Roland's infiniband.git > for_2_6_27 tree. Since 2.6.27-rc1 will be merged into ofed-1.4, I > shouldn't worry about generating patches for ofed-1.4, right? > > It will be in OFED 1.4 You can start preparing the backport patches if needed so the integration will be fater Tziporet. From tziporet at dev.mellanox.co.il Tue Jul 1 02:26:12 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 01 Jul 2008 12:26:12 +0300 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (June 30, 2008) In-Reply-To: <4868F1A3.1030708@opengridcomputing.com> References: <40FA0A8088E8A441973D37502F00933E2B498C@mtlexch01.mtl.com> <4868F1A3.1030708@opengridcomputing.com> Message-ID: <4869F834.3000003@mellanox.co.il> Steve Wise wrote: > The MEM_MGT_EXTENSIONS patches are in Roland's infiniband.git > for_2_6_27 tree. Since 2.6.27-rc1 will be merged into ofed-1.4, I > shouldn't worry about generating patches for ofed-1.4, right? > > It will be in OFED 1.4 You can start preparing the backport patches if needed so the integration will be faster Tziporet. From vlad at lists.openfabrics.org Tue Jul 1 02:44:39 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 1 Jul 2008 02:44:39 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080701-0200 daily build status Message-ID: <20080701094439.D3F87E6094C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c: At top level: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: error: unknown field 'change_queue_depth' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080701-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From eli at dev.mellanox.co.il Tue Jul 1 02:45:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 1 Jul 2008 12:45:25 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change of mtu to use kernel API Message-ID: <20080701094525.GA14384@mtls03> >From aefedbdec99b2db70a0f0de50b9bfb58ccb7e9ab Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 30 Jun 2008 18:59:54 +0300 Subject: [PATCH] IB/IPoIB: Fix change of mtu to use kernel API When the driver sets the MTU of the net device, it should make use of dev_set_mtu() instead of directly setting the mtu field of struct netdevice. Failure to do so will result in functions registered to be called upon MTU change will not get called (this is done through call_netdevice_notifiers()). We have seen problems with performance degredation upon changing the MTU with ofed which has a patch to set the MTU when changing to CM mode to its maximum allowed value. Using dev_set_mtu() solved that problem. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 6223fc3..82f3f2f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1410,7 +1410,7 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, if (!strcmp(buf, "datagram\n")) { clear_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); - dev->mtu = min(priv->mcast_mtu, dev->mtu); + dev_set_mtu(dev, min(priv->mcast_mtu, dev->mtu)) ipoib_flush_paths(dev); if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 4a6538b..be10d78 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -574,7 +574,7 @@ void ipoib_mcast_join_task(struct work_struct *work) priv->mcast_mtu = IPOIB_UD_MTU(ib_mtu_enum_to_int(priv->broadcast->mcmember.mtu)); if (!ipoib_cm_admin_enabled(dev)) - dev->mtu = min(priv->mcast_mtu, priv->admin_mtu); + dev_set_mtu(dev, min(priv->mcast_mtu, priv->admin_mtu)); ipoib_dbg_mcast(priv, "successfully joined all multicast groups\n"); -- 1.5.6 From ogerlitz at voltaire.com Tue Jul 1 05:01:53 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 01 Jul 2008 15:01:53 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <20080701071559.GE6024@mtls03> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> <20080701071559.GE6024@mtls03> Message-ID: <486A1CB1.1020904@voltaire.com> Eli Cohen wrote: > I am not calling call_netdevice_notifiers() directly but through dev_set_mtu() and the only other place I see that makes use of dev_set_mtu() is bond_change_mtu() and it does not usertnl_lock()/rtnl_unlock(). Eli, The calls to dev_set_mtu from the bonding driver are from the device .set_mtu function and this means that the caller have taken the appropriate locking needed (set mtu is done on the master which in turn does it on the slaves). Recently, I worked on some change to bonding and throughout this work I learned on the need (must) to call the rtnl locking when invoking a dev_set_x function who further does call_netdevice_notifiers(), see "the correct locking context for the notifier calls (which is RTNL and nothing else)" comment from the bonding maintainer in http://marc.info/?l=linux-netdev&m=121201324611292&w=2 Again, other than the need to add rtnl locking, I think the patch is good and should be merged. Or From ogerlitz at voltaire.com Tue Jul 1 05:08:09 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 01 Jul 2008 15:08:09 +0300 Subject: [ofa-general] Re: [PATCH v4 for-2.6.27] IPOIB: add LRO support. In-Reply-To: References: <4867D58C.5030404@gmail.com> Message-ID: <486A1E29.6080201@voltaire.com> Roland Dreier wrote: > > +extern int ipoib_use_lro; > > +extern int ipoib_lro_max_aggr; Roland, I the past, few times you have commented that as far as its possible to go without mod params, this should be the practice... for example here sysfs entries set by various hotplug mechanisms, or telling the user to upgrade their ethtool package, etc could serve to eliminate the need in mod params. During the review I asked why the default value for max_aggr is 64 and it was ignored, also what we want the default value for use_lro to be? I recall there was once a comment from Dave Miller saying why it would be wise to have it off by default, I will look for that. Or. From tziporet at mellanox.co.il Tue Jul 1 05:54:23 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Tue, 1 Jul 2008 15:54:23 +0300 Subject: [ofa-general] OFED June 30, 2008 meeting sumary Message-ID: <40FA0A8088E8A441973D37502F00933E30A2ED@mtlexch01.mtl.com> OFED meeting summary for June 30, 2008 ====================================== Summary: ======== - Agreed that all compilation problems should be fixed for beginning of next week - July 7 - All new features from kernel 2.6.27 will be part of OFED 1.4 Details: ======== 1. OFED 1.4 status: - Kernel: - Merged with 2.6.26-rc8 - Components with compilation problems: ipath - Qlogic, ehca - IBM, iSER - Voltaire, qlogic_vnic - Qlogic Qlogic and Voltaire plan to fix compilation problems by Monday next week (July 7). IBM - need update from - New components: iSER target - expected next week NFSoRDMA - no update from Jeff yet - RDS: iWARP support will be added for the feature freeze - Userspace status: - uDAPL: - Moved the 1.0 library to the name compat to make it clear that this is only for compatibility - Added socket based CM - for both scalability and interop with Windows - Added UD extensions - for version 2.0 only - MPI: Open MPI: (from Yiftach since Jeff was not at the meeting): Branch for version 1.3 was opened. Should be ready for OFED 1.4 VMAPICH 1.1 and VMAPICH2 1.2 (OSU - Jonathan): Both should be ready with first version to our feature freeze in 2 weeks. - Management: SM: no report (Sasha did not participated) Congestion Control: Added to ibutils, as experimental feature for OFED 1.4 2. Open discussion: - Rupert asked about RHEL5 version that can be used for the interop event. OFED already supports RHEL 5, 5.1 and 5.2 so the decision depend on the inteop decision. Note: All meeting minutes are now places on the web at: http://www.openfabrics.org/txt/documentation/linux/EWG_meeting_minutes/ From hrosenstock at xsigo.com Tue Jul 1 05:57:10 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 01 Jul 2008 05:57:10 -0700 Subject: [ofa-general] libibumad umad_recv and umad_poll In-Reply-To: <000501c8db1a$288470a0$bb37170a@amr.corp.intel.com> References: <000501c8db1a$288470a0$bb37170a@amr.corp.intel.com> Message-ID: <1214917030.3486.238.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-06-30 at 18:31 -0700, Sean Hefty wrote: > I have a couple questions regarding the libibumad interface, and I'm hoping > someone might know the answers to. > > Do any apps call umad_recv()/umad_poll() from multiple threads? I'm > specifically interested in opensm and the diagnostic utilities. OpenSM does not. There's a single receiver thread which calls umad_recv. AFAIK none of the infiniband-diags do either. I can't speak for ibutils and any out of tree consumers. I also haven't looked at Ira's new libibnetdisc yet. > If so, can umad_recv() support a mix of blocking and non-blocking calls > concurrently? Or are all calls from a single app always blocking or always > non-blocking? > > Is the user expected to modify the fd to support non-blocking calls? libibumad does open the fd as non blocking in umad_open_port: if ((fd = open(dev_file, O_RDWR|O_NONBLOCK)) < 0) -- Hal > I'm more concerned with the current, actual usage of the interface, rather than > the theoretical operation. > > Thanks, > - Sean > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From jean-vincent.ficet at bull.net Tue Jul 1 06:18:54 2008 From: jean-vincent.ficet at bull.net (Vincent Ficet) Date: Tue, 01 Jul 2008 15:18:54 +0200 Subject: [ofa-general] Issues with osm_req_set():tid Message-ID: <486A2EBE.2000003@bull.net> Hello, In the function opensm/opensm/osm_req.c:osm_req_set(), I noticed that the transaction Id was generated as follows: ib_net64_t tid; [ ... ] tid = cl_hton64((uint64_t) cl_atomic_inc(&sm->sm_trans_id)); However, the type of sm->sm_trans_id is atomic32_t and atomic32_t is declared as follows in opensm/include/complib/cl_types_osd.h: typedef volatile int32_t atomic32_t; Is the cast made by the compiler from int32_t (32 bit, signed) to a uint64_t (64 bit, unsigned) always safe w.r.t compiler(s) optimisation flags ? If not, it may be necessary to: - define a new atomic64_t type - define a new cl_atomic_inc64 function - use this function directly in osm_req_set() - redefine the OSM_SM_INITIAL_TID_VALUE constant to be 64 bit long. What is your opinion on this ? Cheers, Vincent From eli at dev.mellanox.co.il Tue Jul 1 06:42:29 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 1 Jul 2008 16:42:29 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <486A1CB1.1020904@voltaire.com> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> <20080701071559.GE6024@mtls03> <486A1CB1.1020904@voltaire.com> Message-ID: <20080701134229.GC14384@mtls03> On Tue, Jul 01, 2008 at 03:01:53PM +0300, Or Gerlitz wrote: > The calls to dev_set_mtu from the bonding driver are from the device > .set_mtu function and this means that the caller have taken the appropriate > locking needed (set mtu is done on the master which in turn does it on the > slaves). Recently, I worked on some change to bonding and throughout this > work I learned on the need (must) to call the rtnl locking when invoking a > dev_set_x function who further does call_netdevice_notifiers(), see > > "the correct locking context for the notifier calls (which is RTNL and > nothing else)" > > comment from the bonding maintainer in > http://marc.info/?l=linux-netdev&m=121201324611292&w=2 > I see, though I would expect to see a comment stating this requirement both at the documentation of call_netdevice_notifiers() and that of dev_set_mtu() and any other exported functions that requires this kind of locking. Moreover, in this specific case, it appears that it is not required to take the rtlnl lock -- if it would be a must, I would have experienced a dump_stack() due to ASSERT_RTNL(); in bond_alb_handle_active_change(). The fact that I did not hit such an assert does not mean I may avoid taking the rtnl lock but it appears to me that the issue is not well undrestood. From eli at dev.mellanox.co.il Tue Jul 1 06:45:56 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 1 Jul 2008 16:45:56 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <20080701134229.GC14384@mtls03> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> <20080701071559.GE6024@mtls03> <486A1CB1.1020904@voltaire.com> <20080701134229.GC14384@mtls03> Message-ID: <20080701134556.GD14384@mtls03> On Tue, Jul 01, 2008 at 04:42:29PM +0300, Eli Cohen wrote: > > I see, though I would expect to see a comment stating this requirement > both at the documentation of call_netdevice_notifiers() and that of > dev_set_mtu() and any other exported functions that requires this kind > of locking. > Moreover, in this specific case, it appears that it is not required to > take the rtlnl lock -- if it would be a must, I would have experienced > a dump_stack() due to > > ASSERT_RTNL(); > > in bond_alb_handle_active_change(). Or some other function registered for notifications. > > > The fact that I did not hit such an assert does not mean I may avoid > taking the rtnl lock but it appears to me that the issue is not well > undrestood. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From fenkes at de.ibm.com Tue Jul 1 07:14:19 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Tue, 1 Jul 2008 16:14:19 +0200 Subject: [ofa-general] [PATCH] IB/ehca: Make device table externally visible In-Reply-To: <48499C11.7030504@gmail.com> References: <200806061835.43802.fenkes@de.ibm.com> <48499C11.7030504@gmail.com> Message-ID: <200807011614.19670.fenkes@de.ibm.com> This gives ehca an autogenerated modalias and therefore enables automatic loading. Signed-off-by: Joachim Fenkes --- drivers/infiniband/hw/ehca/ehca_main.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c index 482103e..598844d 100644 --- a/drivers/infiniband/hw/ehca/ehca_main.c +++ b/drivers/infiniband/hw/ehca/ehca_main.c @@ -923,6 +923,7 @@ static struct of_device_id ehca_device_table[] = }, {}, }; +MODULE_DEVICE_TABLE(of, ehca_device_table); static struct of_platform_driver ehca_driver = { .name = "ehca", -- 1.5.5 From swise at opengridcomputing.com Tue Jul 1 07:15:17 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 09:15:17 -0500 Subject: [ofa-general] [PATCH RFC] RDMA/cxgb3: Fixes for zero-stag. In-Reply-To: <4869BB7E.8050600@voltaire.com> References: <20080626225945.27989.73122.stgit@dell3.ogc.int> <48672023.9050108@voltaire.com> <4867B28D.5090603@opengridcomputing.com> <48686660.9020001@voltaire.com> <4868F6BA.4070601@opengridcomputing.com> <48691081.5080907@opengridcomputing.com> <4869BB7E.8050600@voltaire.com> Message-ID: <486A3BF5.7040708@opengridcomputing.com> Or Gerlitz wrote: > Steve Wise wrote: >> NFSRDMA was planning on using the zero stag for recvs on iwarp >> transports. This boils down to only needing the zero stag + fastreg >> mrs for NFSRDMA server over T3, avoiding dma_mrs which have limits in >> the T3 HW. >>> the zero STag breaks the semantics slightly in that there is no >>> longer any PD attached to the key returned by ib_get_dma_mr > Steve, > > I am fine with the approach of new verb, however, I still want to make > sure that this can be used under a unified IB/iWARP design of a ULP > without much pain... the kernel ULPs: rNFS, iSER initiator (Lustre, > RDS, etc) use the dma mr for recv nd send work-requests. > > So how you envision the ULP code in the presence of the new verb - is > it the ULP to check if the device supports the capability and if yes, > use dev->local_dma_lkey else call get_dma_mr and use the returned lkey? > Yes. > in you comment above you say "using the zero stag for recvs on iwarp > transports" does zero stag applies also to sends? if not, will sends > use dma-mr? > > Yes, NFSRDMA server will use this local_dma_lkey and/or a dma_mr for local access SGES. That includes recvs, sends, and the source of writes. Steve. From swise at opengridcomputing.com Tue Jul 1 07:16:31 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 09:16:31 -0500 Subject: [ofa-general] Re: [PATCH REPOST RFCv2 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <4869BC47.7080405@voltaire.com> References: <20080630175112.15731.20476.stgit@dell3.ogc.int> <20080630175301.15731.47648.stgit@dell3.ogc.int> <4869BC47.7080405@voltaire.com> Message-ID: <486A3C3F.7020503@opengridcomputing.com> Or Gerlitz wrote: > Roland Dreier wrote: >> > --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c >> > + dev->ibdev.local_dma_lkey = 0; >> >> why this specific initialization but not in nes or amso1100? I think >> the device structure is zeroed out on allocation anyway. > To prevent confusion, lets have it documented somewhere that in the > iwarp case IB_DEVICE_LOCAL_DMA_LKEY refers to zero-stag and as the > name suggest its an lkey which equals zero, how does this sound? > I don't see the benefit. Folks can read the RDMA Consortium verbs if they want to understand about 0 stag. Steve. From monis at Voltaire.COM Tue Jul 1 07:36:34 2008 From: monis at Voltaire.COM (Moni Shoua) Date: Tue, 01 Jul 2008 17:36:34 +0300 Subject: [ofa-general] [PATCH V2] IB/ipoib: refresh paths instead of fushing them on SM change event In-Reply-To: References: <484F8A20.9070101@Voltaire.COM> <48624168.3050305@Voltaire.COM> Message-ID: <486A40F2.2060306@Voltaire.COM> Thanks Roland, I refered to your comment below. I will resend V3 with hope that my answers are acceptable and that there are no more issues left. Roland Dreier wrote: > I wanted to apply this but I see some locking issues; > > > @@ -409,6 +427,9 @@ static void path_rec_completion(int status, > > > > spin_lock_irqsave(&priv->lock, flags); > > > > + if (path->ah) > > + ipoib_put_ah(path->ah); > > Look at what ipoib_free_ah() does if this drops the last reference to > the ah (hint -- it takes priv->lock). Is there any guarantee that there > are more references to this ah still held? If so there needs to be a > comment explaining this (as other places in ipoib_main.c have). if path->ah is not NULL then it means that this is not the first visit in path_rec_completion() for this path and we are sure that there is at least one more reference held by a neighbour in this path. I added this comment to the code. > > > list_for_each_entry_safe(neigh, tn, &path->neigh_list, list) { > > + if (neigh->ah) > > + old_ah = neigh->ah; > > is there any guarantee that this list has only one entry? If so why? You are right, there isn't but it seems easy to fix. I don't need to remember all old_ah's but only one that might be the last one that holds the reference and put it outside the lock. I changed this to list_for_each_entry_safe(neigh, tn, &path->neigh_list, list) { + if (neigh->ah) { + /* + * We don't need to remember all old_ah's but + * only the last one that may hold the last + * reference + */ + if (old_ah) + ipoib_put_ah(old_ah); + old_ah = neigh->ah; + } > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From monis at Voltaire.COM Tue Jul 1 07:39:23 2008 From: monis at Voltaire.COM (Moni Shoua) Date: Tue, 01 Jul 2008 17:39:23 +0300 Subject: [ofa-general] [PATCH V3] IB/ipoib: refresh paths instead of fushing them on SM change event In-Reply-To: <484F8A20.9070101@Voltaire.COM> References: <484F8A20.9070101@Voltaire.COM> Message-ID: <486A419B.9010904@Voltaire.COM> The patch tries to solve the problem of device going down and paths being flushed on SM change event. The method is to mark the paths as candidates for refresh (with a valid flag) and wait for ARP probe to start a flow of path lookup that leads to patch query which ends up in path (and address handle) refresh. The solution requires a different and less intrusive handling of SM change event. For that, the second argument of the flush function changed it's meaning from boolean flag to a level. In most cases, SM failover doesn't cause LID change so traffic won't stop. In the rare cases of LID change, the remote host (the one that hadn't changed its LID) will lose connectivity until paths are refreshed. This is no worse than the current state. In fact, preventing the device from going down saves packets that otherwise would be lost. Signed-off-by: Moni Levy Signed-off-by: Moni Shoua --- drivers/infiniband/ulp/ipoib/ipoib.h | 17 +++++++-- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 42 +++++++++++++++-------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 52 ++++++++++++++++++++++++++--- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 19 +++++----- 4 files changed, 98 insertions(+), 32 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index ca126fc..bb24e34 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -55,6 +55,12 @@ /* constants */ +enum ipoib_flush_level { + IPOIB_FLUSH_LIGHT, + IPOIB_FLUSH_NORMAL, + IPOIB_FLUSH_HEAVY +}; + enum { IPOIB_ENCAP_LEN = 4, @@ -276,10 +282,11 @@ struct ipoib_dev_priv { struct delayed_work pkey_poll_task; struct delayed_work mcast_task; - struct work_struct flush_task; + struct work_struct flush_light; + struct work_struct flush_normal; + struct work_struct flush_heavy; struct work_struct restart_task; struct delayed_work ah_reap_task; - struct work_struct pkey_event_task; struct ib_device *ca; u8 port; @@ -358,6 +365,7 @@ struct ipoib_path { struct completion done; struct rb_node rb_node; + int valid; struct list_head list; }; @@ -423,11 +431,14 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_ah *address, u32 qpn); void ipoib_reap_ah(struct work_struct *work); +void ipoib_mark_paths_invalid(struct net_device *dev); void ipoib_flush_paths(struct net_device *dev); struct ipoib_dev_priv *ipoib_intf_alloc(const char *format); int ipoib_ib_dev_init(struct net_device *dev, struct ib_device *ca, int port); -void ipoib_ib_dev_flush(struct work_struct *work); +void ipoib_ib_dev_flush_light(struct work_struct *work); +void ipoib_ib_dev_flush_normal(struct work_struct *work); +void ipoib_ib_dev_flush_heavy(struct work_struct *work); void ipoib_pkey_event(struct work_struct *work); void ipoib_ib_dev_cleanup(struct net_device *dev); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index f429bce..146ee31 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -898,7 +898,8 @@ int ipoib_ib_dev_init(struct net_device *dev, struct ib_device *ca, int port) return 0; } -static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, int pkey_event) +static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, + enum ipoib_flush_level level) { struct ipoib_dev_priv *cpriv; struct net_device *dev = priv->dev; @@ -911,7 +912,7 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, int pkey_event) * the parent is down. */ list_for_each_entry(cpriv, &priv->child_intfs, list) - __ipoib_ib_dev_flush(cpriv, pkey_event); + __ipoib_ib_dev_flush(cpriv, level); mutex_unlock(&priv->vlan_mutex); @@ -925,7 +926,7 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, int pkey_event) return; } - if (pkey_event) { + if (level == IPOIB_FLUSH_HEAVY) { if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &new_index)) { clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); ipoib_ib_dev_down(dev, 0); @@ -943,11 +944,15 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, int pkey_event) priv->pkey_index = new_index; } - ipoib_dbg(priv, "flushing\n"); + if (level == IPOIB_FLUSH_LIGHT) { + ipoib_mark_paths_invalid(dev); + ipoib_mcast_dev_flush(dev); + } - ipoib_ib_dev_down(dev, 0); + if (level >= IPOIB_FLUSH_NORMAL) + ipoib_ib_dev_down(dev, 0); - if (pkey_event) { + if (level == IPOIB_FLUSH_HEAVY) { ipoib_ib_dev_stop(dev, 0); ipoib_ib_dev_open(dev); } @@ -957,27 +962,34 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv, int pkey_event) * we get here, don't bring it back up if it's not configured up */ if (test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) { - ipoib_ib_dev_up(dev); + if (level >= IPOIB_FLUSH_NORMAL) + ipoib_ib_dev_up(dev); ipoib_mcast_restart_task(&priv->restart_task); } } -void ipoib_ib_dev_flush(struct work_struct *work) +void ipoib_ib_dev_flush_light(struct work_struct *work) +{ + struct ipoib_dev_priv *priv = + container_of(work, struct ipoib_dev_priv, flush_light); + + __ipoib_ib_dev_flush(priv, IPOIB_FLUSH_LIGHT); +} + +void ipoib_ib_dev_flush_normal(struct work_struct *work) { struct ipoib_dev_priv *priv = - container_of(work, struct ipoib_dev_priv, flush_task); + container_of(work, struct ipoib_dev_priv, flush_normal); - ipoib_dbg(priv, "Flushing %s\n", priv->dev->name); - __ipoib_ib_dev_flush(priv, 0); + __ipoib_ib_dev_flush(priv, IPOIB_FLUSH_NORMAL); } -void ipoib_pkey_event(struct work_struct *work) +void ipoib_ib_dev_flush_heavy(struct work_struct *work) { struct ipoib_dev_priv *priv = - container_of(work, struct ipoib_dev_priv, pkey_event_task); + container_of(work, struct ipoib_dev_priv, flush_heavy); - ipoib_dbg(priv, "Flushing %s and restarting its QP\n", priv->dev->name); - __ipoib_ib_dev_flush(priv, 1); + __ipoib_ib_dev_flush(priv, IPOIB_FLUSH_HEAVY); } void ipoib_ib_dev_cleanup(struct net_device *dev) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 2442090..5cf5ffb 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -350,6 +350,23 @@ void ipoib_path_iter_read(struct ipoib_path_iter *iter, #endif /* CONFIG_INFINIBAND_IPOIB_DEBUG */ +void ipoib_mark_paths_invalid(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct ipoib_path *path, *tp; + spin_lock_irq(&priv->tx_lock); + spin_lock(&priv->lock); + + list_for_each_entry_safe(path, tp, &priv->path_list, list) { + ipoib_dbg(priv, "mark path LID 0x%04x GID " IPOIB_GID_FMT " invalid\n", + be16_to_cpu(path->pathrec.dlid), + IPOIB_GID_ARG(path->pathrec.dgid)); + path->valid = 0; + } + spin_unlock(&priv->lock); + spin_unlock_irq(&priv->tx_lock); +} + void ipoib_flush_paths(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -386,6 +403,7 @@ static void path_rec_completion(int status, struct net_device *dev = path->dev; struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_ah *ah = NULL; + struct ipoib_ah *old_ah = NULL; struct ipoib_neigh *neigh, *tn; struct sk_buff_head skqueue; struct sk_buff *skb; @@ -409,6 +427,15 @@ static void path_rec_completion(int status, spin_lock_irqsave(&priv->lock, flags); + /* + * if path->ah is not NULL then it means that this is not the first + * visit in path_rec_completion() for this path and we are sure + * that there is at least one more reference held by a neighbour + * in this path + */ + if (path->ah) + ipoib_put_ah(path->ah); + path->ah = ah; if (ah) { @@ -421,6 +448,16 @@ static void path_rec_completion(int status, __skb_queue_tail(&skqueue, skb); list_for_each_entry_safe(neigh, tn, &path->neigh_list, list) { + if (neigh->ah) { + /* + * We don't need to remember all old_ah's but + * only the last one that may hold the last + * reference + */ + if (old_ah) + ipoib_put_ah(old_ah); + old_ah = neigh->ah; + } kref_get(&path->ah->ref); neigh->ah = path->ah; memcpy(&neigh->dgid.raw, &path->pathrec.dgid.raw, @@ -443,13 +480,15 @@ static void path_rec_completion(int status, while ((skb = __skb_dequeue(&neigh->queue))) __skb_queue_tail(&skqueue, skb); } + path->valid = 1; } path->query = NULL; complete(&path->done); spin_unlock_irqrestore(&priv->lock, flags); - + if (old_ah) + ipoib_put_ah(old_ah); while ((skb = __skb_dequeue(&skqueue))) { skb->dev = dev; if (dev_queue_xmit(skb)) @@ -471,6 +510,7 @@ static struct ipoib_path *path_rec_create(struct net_device *dev, void *gid) return NULL; path->dev = dev; + path->valid = 0; skb_queue_head_init(&path->queue); @@ -623,8 +663,9 @@ static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev, spin_lock(&priv->lock); path = __path_find(dev, phdr->hwaddr + 4); - if (!path) { - path = path_rec_create(dev, phdr->hwaddr + 4); + if (!path || !path->valid) { + if (!path) + path = path_rec_create(dev, phdr->hwaddr + 4); if (path) { /* put pseudoheader back on for next time */ skb_push(skb, sizeof *phdr); @@ -989,9 +1030,10 @@ static void ipoib_setup(struct net_device *dev) INIT_LIST_HEAD(&priv->multicast_list); INIT_DELAYED_WORK(&priv->pkey_poll_task, ipoib_pkey_poll); - INIT_WORK(&priv->pkey_event_task, ipoib_pkey_event); INIT_DELAYED_WORK(&priv->mcast_task, ipoib_mcast_join_task); - INIT_WORK(&priv->flush_task, ipoib_ib_dev_flush); + INIT_WORK(&priv->flush_light, ipoib_ib_dev_flush_light); + INIT_WORK(&priv->flush_normal, ipoib_ib_dev_flush_normal); + INIT_WORK(&priv->flush_heavy, ipoib_ib_dev_flush_heavy); INIT_WORK(&priv->restart_task, ipoib_mcast_restart_task); INIT_DELAYED_WORK(&priv->ah_reap_task, ipoib_reap_ah); } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 8766d29..4ab06ea 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -289,15 +289,16 @@ void ipoib_event(struct ib_event_handler *handler, if (record->element.port_num != priv->port) return; - if (record->event == IB_EVENT_PORT_ERR || - record->event == IB_EVENT_PORT_ACTIVE || - record->event == IB_EVENT_LID_CHANGE || - record->event == IB_EVENT_SM_CHANGE || - record->event == IB_EVENT_CLIENT_REREGISTER) { - ipoib_dbg(priv, "Port state change event\n"); - queue_work(ipoib_workqueue, &priv->flush_task); + ipoib_dbg(priv, "Event %d on device %s port %d\n", record->event, + record->device->name, record->element.port_num); + if (record->event == IB_EVENT_SM_CHANGE || + record->event == IB_EVENT_CLIENT_REREGISTER) { + queue_work(ipoib_workqueue, &priv->flush_light); + } else if (record->event == IB_EVENT_PORT_ERR || + record->event == IB_EVENT_PORT_ACTIVE || + record->event == IB_EVENT_LID_CHANGE) { + queue_work(ipoib_workqueue, &priv->flush_normal); } else if (record->event == IB_EVENT_PKEY_CHANGE) { - ipoib_dbg(priv, "P_Key change event on port:%d\n", priv->port); - queue_work(ipoib_workqueue, &priv->pkey_event_task); + queue_work(ipoib_workqueue, &priv->flush_heavy); } } From swise at opengridcomputing.com Tue Jul 1 07:44:41 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 09:44:41 -0500 Subject: [ofa-general] Re: [PATCH REPOST RFCv2 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <486A3C3F.7020503@opengridcomputing.com> References: <20080630175112.15731.20476.stgit@dell3.ogc.int> <20080630175301.15731.47648.stgit@dell3.ogc.int> <4869BC47.7080405@voltaire.com> <486A3C3F.7020503@opengridcomputing.com> Message-ID: <486A42D9.9010704@opengridcomputing.com> Steve Wise wrote: > Or Gerlitz wrote: >> Roland Dreier wrote: >>> > --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c >>> > + dev->ibdev.local_dma_lkey = 0; >>> >>> why this specific initialization but not in nes or amso1100? I >>> think the device structure is zeroed out on allocation anyway. >> To prevent confusion, lets have it documented somewhere that in the >> iwarp case IB_DEVICE_LOCAL_DMA_LKEY refers to zero-stag and as the >> name suggest its an lkey which equals zero, how does this sound? >> > > I don't see the benefit. Folks can read the RDMA Consortium verbs if > they want to understand about 0 stag. > > Steve. > I can add a comment in the code above if you think that's necessary. Something like: /* * Use stag 0 for the global local access lkey. */ From weiny2 at llnl.gov Tue Jul 1 09:08:06 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 1 Jul 2008 09:08:06 -0700 Subject: [ofa-general] libibumad umad_recv and umad_poll In-Reply-To: <1214917030.3486.238.camel@hrosenstock-ws.xsigo.com> References: <000501c8db1a$288470a0$bb37170a@amr.corp.intel.com> <1214917030.3486.238.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080701090806.74e14bda.weiny2@llnl.gov> On Tue, 01 Jul 2008 05:57:10 -0700 Hal Rosenstock wrote: > On Mon, 2008-06-30 at 18:31 -0700, Sean Hefty wrote: > > I have a couple questions regarding the libibumad interface, and I'm hoping > > someone might know the answers to. > > > > Do any apps call umad_recv()/umad_poll() from multiple threads? I'm > > specifically interested in opensm and the diagnostic utilities. > > OpenSM does not. There's a single receiver thread which calls umad_recv. > AFAIK none of the infiniband-diags do either. > > I can't speak for ibutils and any out of tree consumers. > > I also haven't looked at Ira's new libibnetdisc yet. I make the same calls ibnetdiscover does, for now... I have thought that using multiple threads would make some things faster but it seems fast enough for now. Ira > > > If so, can umad_recv() support a mix of blocking and non-blocking calls > > concurrently? Or are all calls from a single app always blocking or always > > non-blocking? > > > > Is the user expected to modify the fd to support non-blocking calls? > > libibumad does open the fd as non blocking in umad_open_port: > if ((fd = open(dev_file, O_RDWR|O_NONBLOCK)) < 0) > > -- Hal > > > I'm more concerned with the current, actual usage of the interface, rather than > > the theoretical operation. > > > > Thanks, > > - Sean > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sean.hefty at intel.com Tue Jul 1 09:32:31 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 1 Jul 2008 09:32:31 -0700 Subject: [ofa-general] libibumad umad_recv and umad_poll In-Reply-To: <20080701090806.74e14bda.weiny2@llnl.gov> References: <000501c8db1a$288470a0$bb37170a@amr.corp.intel.com><1214917030.3486.238.camel@hrosenstock-ws.xsigo.com> <20080701090806.74e14bda.weiny2@llnl.gov> Message-ID: <000a01c8db98$0f8e6000$e1c8180a@amr.corp.intel.com> Ok - thanks for the info. I think this is enough for me to go on. - Sean From joe at perches.com Tue Jul 1 09:49:46 2008 From: joe at perches.com (Joe Perches) Date: Tue, 01 Jul 2008 09:49:46 -0700 Subject: [ofa-general] [RFC PATCH] - scripts/CodingStyle_sizeof - standardize sizeof and [PATCH] drivers/infiniband - standardize sizeof(foo) Message-ID: <1214930986.26073.40.camel@localhost> Scripts could be used to generate CodingStyle patches in reviewable units. For instance, kernel source has multiple style uses of sizeof sizeof(foo) sizeof foo sizeof (foo) Perhaps Julia Lawall semantic patches might work better, but this script might help standardize on the CodingStyle preferred sizeof(foo). It's an imperfect tool. Any patch generated by a tool (yes, I just called myself a tool..:) must be inspected and verified before being applied. A sample patch of drivers_infiniband.patch generated by $ scripts/CodingStyle_sizeof drivers/infiniband is also attached as bz2 Signed-off-by: Joe Perches --- scripts/CodingStyle_sizeof | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-) diff --git a/scripts/CodingStyle_sizeof b/scripts/CodingStyle_sizeof new file mode 100755 index 0000000..7f2444f --- /dev/null +++ b/scripts/CodingStyle_sizeof @@ -0,0 +1,33 @@ +#!/bin/bash +if [ $# -ne 1 ] ; then + echo "$0 takes 1 argument: directory|file" + exit +fi +if [[ ! -f $1 && ! -d $1 ]] ; then + echo "Directory or file: $1 not found" + exit +fi +if [ -d $1 ] ; then + findfile="find $1 -name \"*.[ch]\"" +else + findfile="ls $1" +fi +patchfile=$1.patch +patchfile=${patchfile//\//_} +git checkout $1 +for file in $(eval $findfile) ; do + sed -r -i -e 's/\bsizeof\s+([^\(][^ ;,]+)\s*/sizeof\(\1\)/g' $file + sed -r -i -e 's/\bsizeof\s+/sizeof/g' $file +done +git diff -p --stat $1 > $patchfile +# fixup the generated diff +# add space between close paren and (brace, assign, arithmetic or logical test) +sed -r -i -e 's/^\+(.*)\)([\{\=\+\*\/\<\>\|\!\&\?\-])/\+\1\) \2/g' $patchfile +# remove space between ') ->' added by sed above +sed -r -i -e 's/^\+(.*)\) ->/\+\1\)->/g' $patchfile +# fix declaration with sizeof brackets ie: char foo[sizeof("bar")]; +sed -r -i -e 's/^\+(.*)\[(.*)\((.*)\]\)/\+\1\[\2\(\3\)\]/g' $patchfile +# fix sizeof array element ie: sizeof(foo->bar[n]) +sed -r -i -e 's/^\+(.*)sizeof\((.*)\[(.*)\)\]/\+\1sizeof\(\2\[\3\]\)/g' $patchfile +git checkout $1 +patch -p1 < $patchfile -------------- next part -------------- A non-text attachment was scrubbed... Name: drivers_infiniband.patch.bz2 Type: application/x-bzip Size: 37502 bytes Desc: not available URL: From rdreier at cisco.com Tue Jul 1 10:53:29 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 10:53:29 -0700 Subject: [ofa-general] Re: [PATCH v4 for-2.6.27] IPOIB: add LRO support. In-Reply-To: <486A1E29.6080201@voltaire.com> (Or Gerlitz's message of "Tue, 01 Jul 2008 15:08:09 +0300") References: <4867D58C.5030404@gmail.com> <486A1E29.6080201@voltaire.com> Message-ID: > I the past, few times you have commented that as far as its possible > to go without mod params, this should be the practice... for example > here sysfs entries set by various hotplug mechanisms, or telling the > user to upgrade their ethtool package, etc could serve to eliminate > the need in mod params. > During the review I asked why the default value for max_aggr is 64 and > it was ignored, also what we want the default value for use_lro to be? > I recall there was once a comment from Dave Miller saying why it would > be wise to have it off by default, I will look for that. Yes, in general I prefer not to have module parameters, especially those that duplicate functionality available through other mechanisms. However, having lro and max_aggr parameters seems to match existing practice for ethernet drivers, and 64 seems to be the common default for max_aggr. - R. From chu11 at llnl.gov Tue Jul 1 10:54:01 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 01 Jul 2008 10:54:01 -0700 Subject: [ofa-general] [OPENSM][TRIVIAL] fix true/false usage Message-ID: <1214934841.16508.19.camel@cardanus.llnl.gov> Hey Sasha, Just noticed that true/false was not used consistently for some variables. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-fix-true-false-usage.patch Type: text/x-patch Size: 2356 bytes Desc: not available URL: From rdreier at cisco.com Tue Jul 1 10:55:09 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 10:55:09 -0700 Subject: [ofa-general] Re: [PATCH] IB/ehca: Make device table externally visible In-Reply-To: <200807011614.19670.fenkes@de.ibm.com> (Joachim Fenkes's message of "Tue, 1 Jul 2008 16:14:19 +0200") References: <200806061835.43802.fenkes@de.ibm.com> <48499C11.7030504@gmail.com> <200807011614.19670.fenkes@de.ibm.com> Message-ID: thanks, applied From rdreier at cisco.com Tue Jul 1 10:57:07 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 10:57:07 -0700 Subject: [ofa-general] Re: [RFC PATCH] - scripts/CodingStyle_sizeof - standardize sizeof and [PATCH] drivers/infiniband - standardize sizeof(foo) In-Reply-To: <1214930986.26073.40.camel@localhost> (Joe Perches's message of "Tue, 01 Jul 2008 09:49:46 -0700") References: <1214930986.26073.40.camel@localhost> Message-ID: This seems like needless churn to me. I personally prefer to see "sizeof foo" instead of "sizeof(foo)", since sizeof is not a function, and in any case I don't see much value in making this type of trivial change. From joe at perches.com Tue Jul 1 10:37:44 2008 From: joe at perches.com (Joe Perches) Date: Tue, 01 Jul 2008 10:37:44 -0700 Subject: [ofa-general] ***SPAM*** Re: [RFC PATCH] - scripts/CodingStyle_sizeof - standardize sizeof and [PATCH] drivers/infiniband - standardize sizeof(foo) In-Reply-To: References: <1214930986.26073.40.camel@localhost> Message-ID: <1214933864.26073.50.camel@localhost> On Tue, 2008-07-01 at 10:57 -0700, Roland Dreier wrote: > This seems like needless churn to me. I personally prefer to see > "sizeof foo" instead of "sizeof(foo)", since sizeof is not a function, > and in any case I don't see much value in making this type of trivial > change. No value but standardization and ease of grep. fyi: drivers/infiniband uses 3 different styles for sizeof. Your preferred style is not the most frequently used. $ grep -Pr --include=*.[ch] "\bsizeof\(" drivers/infiniband | wc -l 785 $ grep -Pr --include=*.[ch] "\bsizeof\s+\(" drivers/infiniband | wc -l 226 $ grep -Pr --include=*.[ch] "\bsizeof\s+[^\(]" drivers/infiniband | wc -l 523 cheers, Joe From dotanba at gmail.com Tue Jul 1 12:21:37 2008 From: dotanba at gmail.com (Dotan Barak) Date: Tue, 01 Jul 2008 21:21:37 +0200 Subject: [ofa-general] Re: [RFC PATCH] - scripts/CodingStyle_sizeof - standardize sizeof and [PATCH] drivers/infiniband - standardize sizeof(foo) In-Reply-To: References: <1214930986.26073.40.camel@localhost> Message-ID: <486A83C1.9090803@gmail.com> Roland Dreier wrote: > This seems like needless churn to me. I personally prefer to see > "sizeof foo" instead of "sizeof(foo)", since sizeof is not a function, > and in any case I don't see much value in making this type of trivial > change. > In the IB stack i noticed all of the variations of sizeof: sizeof foo or sizeof(foo) (and i think that i even saw) sizeof (foo) IMHO, Changing all of the instances of "sizeof" from all of the variations to a unified one can make the code look a little bit better ... Dotan From chu11 at llnl.gov Tue Jul 1 11:46:51 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 01 Jul 2008 11:46:51 -0700 Subject: [ofa-general] [OPENSM][TRIVIAL] Fix comment typo Message-ID: <1214938011.16508.21.camel@cardanus.llnl.gov> Hey Sasha, Typo I noticed. Nothing fancy. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-fix-typo-force_immediate_heavy_sweep-is-now-force_.patch Type: text/x-patch Size: 1274 bytes Desc: not available URL: From chu11 at llnl.gov Tue Jul 1 11:46:51 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 01 Jul 2008 11:46:51 -0700 Subject: [ofa-general] [OPENSM][TRIVIAL] Fix comment typo Message-ID: <1214938011.16508.21.camel@cardanus.llnl.gov> Hey Sasha, Typo I noticed. Nothing fancy. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-fix-typo-force_immediate_heavy_sweep-is-now-force_.patch Type: text/x-patch Size: 1274 bytes Desc: not available URL: From reactivation at google.com Tue Jul 1 12:05:53 2008 From: reactivation at google.com (Google AdWords) Date: Tue, 1 Jul 2008 14:05:53 -0500 Subject: [ofa-general] Please Update Your Billing Information. Message-ID: <01c8db83$92ca7680$539d90cf@fionaimrie> ---------------------------------------------------------------- ---------------------------------------------------------- Dear Advertiser, We were unable to process your payment. Your ads will be suspended soon unless we can process your payment. To prevent your ads from being suspended, please update your payment information. Please sign in to your account and update your payment information. We look forward to providing you with the most effective advertising available. Thank you for advertising with Google AdWords. The Google AdWords Team. ------------------------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon at opengridcomputing.com Tue Jul 1 13:00:25 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Tue, 1 Jul 2008 15:00:25 -0500 Subject: [ofa-general] [PATCH] cxgb3: Propigate HW Page Size Capabilities Message-ID: <20080701200024.GA19021@opengridcomputing.com> cxgb3 does not currently report the page size capabilities, and incorrectly reports them internally (due to a lack of shifting). This patch fixes this. Thanks, Jon Signed-Off-By: Jon Mason diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.h b/drivers/infiniband/hw/cxgb3/cxio_hal.h index 25a8806..f2af9cd 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_hal.h +++ b/drivers/infiniband/hw/cxgb3/cxio_hal.h @@ -54,6 +54,7 @@ #define T3_MAX_RQ_SIZE 1024 #define T3_MAX_NUM_STAG (1<<15) #define T3_MAX_MR_SIZE 0x100000000ULL +#define T3_PAGESIZE_MASK (0x7fff << 12) /* 4KB-128MB */ #define T3_STAG_UNSET 0xffffffff diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c index 71554ea..04b2f57 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.c +++ b/drivers/infiniband/hw/cxgb3/iwch.c @@ -82,7 +82,7 @@ static void rnic_init(struct iwch_dev *rnicp) rnicp->attr.max_mem_regs = cxio_num_stags(&rnicp->rdev); rnicp->attr.max_phys_buf_entries = T3_MAX_PBL_SIZE; rnicp->attr.max_pds = T3_MAX_NUM_PD - 1; - rnicp->attr.mem_pgsizes_bitmask = 0x7FFF; /* 4KB-128MB */ + rnicp->attr.mem_pgsizes_bitmask = T3_PAGESIZE_MASK; rnicp->attr.max_mr_size = T3_MAX_MR_SIZE; rnicp->attr.can_resize_wq = 0; rnicp->attr.max_rdma_reads_per_qp = 8; diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index dfc4d1e..264007e 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1103,6 +1103,7 @@ static int iwch_query_device(struct ib_device *ibdev, memset(props, 0, sizeof *props); memcpy(&props->sys_image_guid, dev->rdev.t3cdev_p->lldev->dev_addr, 6); props->device_cap_flags = dev->device_cap_flags; + props->page_size_cap = dev->attr.mem_pgsizes_bitmask; props->vendor_id = (u32)dev->rdev.rnic_info.pdev->vendor; props->vendor_part_id = (u32)dev->rdev.rnic_info.pdev->device; props->max_mr_size = dev->attr.max_mr_size; From swise at opengridcomputing.com Tue Jul 1 13:15:02 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 15:15:02 -0500 Subject: [ofa-general] [PATCH for-2.6.27 0/2] RDMA: Local dma lkey support. Message-ID: <20080701201502.20859.88618.stgit@dell3.ogc.int> Roland, I think this is ready. I moved the location of the new local_dma_lkey to not leave alignment holes. And added a comment for Or's request. -- Steve. From swise at opengridcomputing.com Tue Jul 1 13:17:08 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 15:17:08 -0500 Subject: [ofa-general] [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <20080701201502.20859.88618.stgit@dell3.ogc.int> References: <20080701201502.20859.88618.stgit@dell3.ogc.int> Message-ID: <20080701201708.20859.49697.stgit@dell3.ogc.int> From: Steve Wise - Change the IB_DEVICE_ZERO_STAG flag to the transport-neutral name IB_DEVICE_LOCAL_DMA_LKEY. - Add local_dma_lkey u32 to the ib_device struct. Drivers will fill this in with the appropriate local dma lkey if they support it. - Fix up the drivers using this flag. --- drivers/infiniband/hw/amso1100/c2_rnic.c | 2 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 8 ++++++-- drivers/infiniband/hw/nes/nes_hw.c | 2 +- include/rdma/ib_verbs.h | 3 ++- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/amso1100/c2_rnic.c b/drivers/infiniband/hw/amso1100/c2_rnic.c index b1441ae..dd05c48 100644 --- a/drivers/infiniband/hw/amso1100/c2_rnic.c +++ b/drivers/infiniband/hw/amso1100/c2_rnic.c @@ -454,7 +454,7 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev) (IB_DEVICE_RESIZE_MAX_WR | IB_DEVICE_CURR_QP_STATE_MOD | IB_DEVICE_SYS_IMAGE_GUID | - IB_DEVICE_ZERO_STAG | + IB_DEVICE_LOCAL_DMA_LKEY | IB_DEVICE_MEM_WINDOW); /* Allocate the qptr_array */ diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index 3101e3c..187878d 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1254,8 +1254,12 @@ int iwch_register_device(struct iwch_dev *dev) memset(&dev->ibdev.node_guid, 0, sizeof(dev->ibdev.node_guid)); memcpy(&dev->ibdev.node_guid, dev->rdev.t3cdev_p->lldev->dev_addr, 6); dev->ibdev.owner = THIS_MODULE; - dev->device_cap_flags = IB_DEVICE_ZERO_STAG | - IB_DEVICE_MEM_WINDOW; + dev->device_cap_flags = IB_DEVICE_LOCAL_DMA_LKEY | IB_DEVICE_MEM_WINDOW; + + /* + * Use stag 0 for local kernel mode access. + */ + dev->ibdev.local_dma_lkey = 0; if (fw_supports_fastreg(dev)) dev->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c index d3278f1..070b0c7 100644 --- a/drivers/infiniband/hw/nes/nes_hw.c +++ b/drivers/infiniband/hw/nes/nes_hw.c @@ -398,7 +398,7 @@ struct nes_adapter *nes_init_adapter(struct nes_device *nesdev, u8 hw_rev) { nesadapter->base_pd = 1; nesadapter->device_cap_flags = - IB_DEVICE_ZERO_STAG | IB_DEVICE_MEM_WINDOW; + IB_DEVICE_LOCAL_DMA_LKEY | IB_DEVICE_MEM_WINDOW; nesadapter->allocated_qps = (unsigned long *)&(((unsigned char *)nesadapter) [(sizeof(struct nes_adapter)+(sizeof(unsigned long)-1))&(~(sizeof(unsigned long)-1))]); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 74c24b9..84aff7e 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -91,7 +91,7 @@ enum ib_device_cap_flags { IB_DEVICE_RC_RNR_NAK_GEN = (1<<12), IB_DEVICE_SRQ_RESIZE = (1<<13), IB_DEVICE_N_NOTIFY_CQ = (1<<14), - IB_DEVICE_ZERO_STAG = (1<<15), + IB_DEVICE_LOCAL_DMA_LKEY = (1<<15), IB_DEVICE_RESERVED = (1<<16), /* old SEND_W_INV */ IB_DEVICE_MEM_WINDOW = (1<<17), /* @@ -1094,6 +1094,7 @@ struct ib_device { char node_desc[64]; __be64 node_guid; + u32 local_dma_lkey; u8 node_type; u8 phys_port_cnt; }; From swise at opengridcomputing.com Tue Jul 1 13:17:10 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 15:17:10 -0500 Subject: [ofa-general] [PATCH for-2.6.27 2/2] RDMA/cxgb3: Fixes for zero-stag. In-Reply-To: <20080701201502.20859.88618.stgit@dell3.ogc.int> References: <20080701201502.20859.88618.stgit@dell3.ogc.int> Message-ID: <20080701201710.20859.39200.stgit@dell3.ogc.int> From: Steve Wise Handling the zero-stag in recv WRs requires some extra logic: - Only set the QP_PRIV bit if it is a kernel mode qp. - Add a zero-stag build function for recv wrs. The uP needs a PBL allocated and passed down in the recv WR so it can construct a hw pbl for the zero-stag sges. Note: we need to place a few restrictions on zero-stag usage: 1) all SGEs in a RECV WR must either be zero-stag or not. No mixing. 2) an individual SGE length cannot exceed 128MB for a zero-stag SGE. This is ok since its not possible to allocate such a large chunk of pinned contiguous dma-coherent memory. - Add an optimized non zero-stag recv wr format for kernel users. This is needed to optimize both zero and non-zero stag cracking in the recv path for kernel users. - Remove the iwch_ prefix from the static build functions. - Bump required fw version. Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/cxio_hal.c | 12 ++- drivers/infiniband/hw/cxgb3/cxio_wr.h | 13 ++- drivers/infiniband/hw/cxgb3/iwch_provider.c | 4 - drivers/infiniband/hw/cxgb3/iwch_qp.c | 127 +++++++++++++++++++++++---- drivers/net/cxgb3/version.h | 2 5 files changed, 131 insertions(+), 27 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.c b/drivers/infiniband/hw/cxgb3/cxio_hal.c index 340e418..9c049ba 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_hal.c +++ b/drivers/infiniband/hw/cxgb3/cxio_hal.c @@ -278,7 +278,7 @@ int cxio_create_qp(struct cxio_rdev *rdev_p, u32 kernel_domain, if (!wq->qpid) return -ENOMEM; - wq->rq = kzalloc(depth * sizeof(u64), GFP_KERNEL); + wq->rq = kzalloc(depth * sizeof(struct t3_swrq), GFP_KERNEL); if (!wq->rq) goto err1; @@ -302,6 +302,7 @@ int cxio_create_qp(struct cxio_rdev *rdev_p, u32 kernel_domain, if (!kernel_domain) wq->udb = (u64)rdev_p->rnic_info.udbell_physbase + (wq->qpid << rdev_p->qpshift); + wq->rdev = rdev_p; PDBG("%s qpid 0x%x doorbell 0x%p udb 0x%llx\n", __func__, wq->qpid, wq->doorbell, (unsigned long long) wq->udb); return 0; @@ -1266,13 +1267,16 @@ proc_cqe: wq->sq_rptr = CQE_WRID_SQ_WPTR(*hw_cqe); PDBG("%s completing sq idx %ld\n", __func__, Q_PTR2IDX(wq->sq_rptr, wq->sq_size_log2)); - *cookie = (wq->sq + - Q_PTR2IDX(wq->sq_rptr, wq->sq_size_log2))->wr_id; + *cookie = wq->sq[Q_PTR2IDX(wq->sq_rptr,wq->sq_size_log2)].wr_id; wq->sq_rptr++; } else { PDBG("%s completing rq idx %ld\n", __func__, Q_PTR2IDX(wq->rq_rptr, wq->rq_size_log2)); - *cookie = *(wq->rq + Q_PTR2IDX(wq->rq_rptr, wq->rq_size_log2)); + *cookie = wq->rq[Q_PTR2IDX(wq->rq_rptr,wq->rq_size_log2)].wr_id; + if (wq->rq[Q_PTR2IDX(wq->rq_rptr, wq->rq_size_log2)].pbl_addr) + cxio_hal_pblpool_free(wq->rdev, + wq->rq[Q_PTR2IDX(wq->rq_rptr, + wq->rq_size_log2)].pbl_addr, T3_STAG0_PBL_SIZE); wq->rq_rptr++; } diff --git a/drivers/infiniband/hw/cxgb3/cxio_wr.h b/drivers/infiniband/hw/cxgb3/cxio_wr.h index 952074f..4a8b7fb 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_wr.h +++ b/drivers/infiniband/hw/cxgb3/cxio_wr.h @@ -39,6 +39,9 @@ #define T3_MAX_SGE 4 #define T3_MAX_INLINE 64 +#define T3_STAG0_PBL_SIZE (2 * T3_MAX_SGE << 3) +#define T3_STAG0_MAX_PBE_LEN (128 * 1024 * 1024) +#define T3_STAG0_PAGE_SHIFT 15 #define Q_EMPTY(rptr,wptr) ((rptr)==(wptr)) #define Q_FULL(rptr,wptr,size_log2) ( (((wptr)-(rptr))>>(size_log2)) && \ @@ -664,6 +667,11 @@ struct t3_swsq { int signaled; }; +struct t3_swrq { + __u64 wr_id; + __u32 pbl_addr; +}; + /* * A T3 WQ implements both the SQ and RQ. */ @@ -680,14 +688,15 @@ struct t3_wq { u32 sq_wptr; /* sq_wptr - sq_rptr == count of */ u32 sq_rptr; /* pending wrs */ u32 sq_size_log2; /* sq size */ - u64 *rq; /* SW RQ (holds consumer wr_ids */ + struct t3_swrq *rq; /* SW RQ (holds consumer wr_ids */ u32 rq_wptr; /* rq_wptr - rq_rptr == count of */ u32 rq_rptr; /* pending wrs */ - u64 *rq_oldest_wr; /* oldest wr on the SW RQ */ + struct t3_swrq *rq_oldest_wr; /* oldest wr on the SW RQ */ u32 rq_size_log2; /* rq size */ u32 rq_addr; /* rq adapter address */ void __iomem *doorbell; /* kernel db */ u64 udb; /* user db if any */ + struct cxio_rdev *rdev; }; struct t3_cq { diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index 187878d..4ee33e7 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1005,10 +1005,10 @@ static struct ib_qp *iwch_create_qp(struct ib_pd *pd, qhp->ibqp.qp_num = qhp->wq.qpid; init_timer(&(qhp->timer)); PDBG("%s sq_num_entries %d, rq_num_entries %d " - "qpid 0x%0x qhp %p dma_addr 0x%llx size %d\n", + "qpid 0x%0x qhp %p dma_addr 0x%llx size %d rq_addr 0x%x\n", __func__, qhp->attr.sq_num_entries, qhp->attr.rq_num_entries, qhp->wq.qpid, qhp, (unsigned long long) qhp->wq.dma_addr, - 1 << qhp->wq.size_log2); + 1 << qhp->wq.size_log2, qhp->wq.rq_addr); return &qhp->ibqp; } diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c index 169056e..30085c7 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_qp.c +++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c @@ -33,10 +33,11 @@ #include "iwch.h" #include "iwch_cm.h" #include "cxio_hal.h" +#include "cxio_resource.h" #define NO_SUPPORT -1 -static int iwch_build_rdma_send(union t3_wr *wqe, struct ib_send_wr *wr, +static int build_rdma_send(union t3_wr *wqe, struct ib_send_wr *wr, u8 * flit_cnt) { int i; @@ -81,7 +82,7 @@ static int iwch_build_rdma_send(union t3_wr *wqe, struct ib_send_wr *wr, return 0; } -static int iwch_build_rdma_write(union t3_wr *wqe, struct ib_send_wr *wr, +static int build_rdma_write(union t3_wr *wqe, struct ib_send_wr *wr, u8 *flit_cnt) { int i; @@ -122,7 +123,7 @@ static int iwch_build_rdma_write(union t3_wr *wqe, struct ib_send_wr *wr, return 0; } -static int iwch_build_rdma_read(union t3_wr *wqe, struct ib_send_wr *wr, +static int build_rdma_read(union t3_wr *wqe, struct ib_send_wr *wr, u8 *flit_cnt) { if (wr->num_sge > 1) @@ -140,7 +141,7 @@ static int iwch_build_rdma_read(union t3_wr *wqe, struct ib_send_wr *wr, return 0; } -static int iwch_build_fastreg(union t3_wr *wqe, struct ib_send_wr *wr, +static int build_fastreg(union t3_wr *wqe, struct ib_send_wr *wr, u8 *flit_cnt, int *wr_cnt, struct t3_wq *wq) { int i; @@ -182,7 +183,7 @@ static int iwch_build_fastreg(union t3_wr *wqe, struct ib_send_wr *wr, return 0; } -static int iwch_build_inv_stag(union t3_wr *wqe, struct ib_send_wr *wr, +static int build_inv_stag(union t3_wr *wqe, struct ib_send_wr *wr, u8 *flit_cnt) { wqe->local_inv.stag = cpu_to_be32(wr->ex.invalidate_rkey); @@ -241,23 +242,106 @@ static int iwch_sgl2pbl_map(struct iwch_dev *rhp, struct ib_sge *sg_list, return 0; } -static int iwch_build_rdma_recv(struct iwch_dev *rhp, union t3_wr *wqe, +static int build_rdma_recv(struct iwch_qp *qhp, union t3_wr *wqe, struct ib_recv_wr *wr) { - int i; - if (wr->num_sge > T3_MAX_SGE) - return -EINVAL; + int i, err = 0; + u32 pbl_addr[T3_MAX_SGE]; + u8 page_size[T3_MAX_SGE]; + + err = iwch_sgl2pbl_map(qhp->rhp, wr->sg_list, wr->num_sge, pbl_addr, + page_size); + if (err) + return err; + wqe->recv.pagesz[0] = page_size[0]; + wqe->recv.pagesz[1] = page_size[1]; + wqe->recv.pagesz[2] = page_size[2]; + wqe->recv.pagesz[3] = page_size[3]; wqe->recv.num_sgle = cpu_to_be32(wr->num_sge); for (i = 0; i < wr->num_sge; i++) { wqe->recv.sgl[i].stag = cpu_to_be32(wr->sg_list[i].lkey); wqe->recv.sgl[i].len = cpu_to_be32(wr->sg_list[i].length); + + /* to in the WQE == the offset into the page */ + wqe->recv.sgl[i].to = cpu_to_be64(((u32) wr->sg_list[i].addr) % + (1UL << (12 + page_size[i]))); + + /* pbl_addr is the adapters address in the PBL */ + wqe->recv.pbl_addr[i] = cpu_to_be32(pbl_addr[i]); + } + for (; i < T3_MAX_SGE; i++) { + wqe->recv.sgl[i].stag = 0; + wqe->recv.sgl[i].len = 0; + wqe->recv.sgl[i].to = 0; + wqe->recv.pbl_addr[i] = 0; + } + qhp->wq.rq[Q_PTR2IDX(qhp->wq.rq_wptr, + qhp->wq.rq_size_log2)].wr_id = wr->wr_id; + qhp->wq.rq[Q_PTR2IDX(qhp->wq.rq_wptr, + qhp->wq.rq_size_log2)].pbl_addr = 0; + return 0; +} + +static int build_zero_stag_recv(struct iwch_qp *qhp, union t3_wr *wqe, + struct ib_recv_wr *wr) +{ + int i; + u32 pbl_addr; + u32 pbl_offset; + + + /* + * The T3 HW requires the PBL in the hw recv descriptor to reference + * a PBL entry. So we allocate the max needed PBL memory here and pass + * it to the uP in the recv WR. The uP will build the PBL and setup + * the hw recv descriptor. + */ + pbl_addr = cxio_hal_pblpool_alloc(&qhp->rhp->rdev, T3_STAG0_PBL_SIZE); + if (!pbl_addr) + return -ENOMEM; + + /* + * Compute the 8B aligned offset. + */ + pbl_offset = (pbl_addr - qhp->rhp->rdev.rnic_info.pbl_base) >> 3; + + wqe->recv.num_sgle = cpu_to_be32(wr->num_sge); + + for (i = 0; i < wr->num_sge; i++) { + + /* + * Use a 128MB page size. This and an imposed 128MB + * sge length limit allows us to require only a 2-entry HW + * PBL for each SGE. This restriction is acceptable since + * since it is not possible to allocate 128MB of contiguous + * DMA coherent memory! + */ + if (wr->sg_list[i].length > T3_STAG0_MAX_PBE_LEN) + return -EINVAL; + wqe->recv.pagesz[i] = T3_STAG0_PAGE_SHIFT; + + /* + * T3 restricts a recv to all zero-stag or all non-zero-stag. + */ + if (wr->sg_list[i].lkey != 0) + return -EINVAL; + wqe->recv.sgl[i].stag = 0; + wqe->recv.sgl[i].len = cpu_to_be32(wr->sg_list[i].length); wqe->recv.sgl[i].to = cpu_to_be64(wr->sg_list[i].addr); + wqe->recv.pbl_addr[i] = cpu_to_be32(pbl_offset); + pbl_offset += 2; } for (; i < T3_MAX_SGE; i++) { + wqe->recv.pagesz[i] = 0; wqe->recv.sgl[i].stag = 0; wqe->recv.sgl[i].len = 0; wqe->recv.sgl[i].to = 0; + wqe->recv.pbl_addr[i] = 0; } + qhp->wq.rq[Q_PTR2IDX(qhp->wq.rq_wptr, + qhp->wq.rq_size_log2)].wr_id = wr->wr_id; + qhp->wq.rq[Q_PTR2IDX(qhp->wq.rq_wptr, + qhp->wq.rq_size_log2)].pbl_addr = pbl_addr; return 0; } @@ -309,17 +393,17 @@ int iwch_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, if (wr->send_flags & IB_SEND_FENCE) t3_wr_flags |= T3_READ_FENCE_FLAG; t3_wr_opcode = T3_WR_SEND; - err = iwch_build_rdma_send(wqe, wr, &t3_wr_flit_cnt); + err = build_rdma_send(wqe, wr, &t3_wr_flit_cnt); break; case IB_WR_RDMA_WRITE: case IB_WR_RDMA_WRITE_WITH_IMM: t3_wr_opcode = T3_WR_WRITE; - err = iwch_build_rdma_write(wqe, wr, &t3_wr_flit_cnt); + err = build_rdma_write(wqe, wr, &t3_wr_flit_cnt); break; case IB_WR_RDMA_READ: t3_wr_opcode = T3_WR_READ; t3_wr_flags = 0; /* T3 reads are always signaled */ - err = iwch_build_rdma_read(wqe, wr, &t3_wr_flit_cnt); + err = build_rdma_read(wqe, wr, &t3_wr_flit_cnt); if (err) break; sqp->read_len = wqe->read.local_len; @@ -328,14 +412,14 @@ int iwch_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, break; case IB_WR_FAST_REG_MR: t3_wr_opcode = T3_WR_FASTREG; - err = iwch_build_fastreg(wqe, wr, &t3_wr_flit_cnt, + err = build_fastreg(wqe, wr, &t3_wr_flit_cnt, &wr_cnt, &qhp->wq); break; case IB_WR_LOCAL_INV: if (wr->send_flags & IB_SEND_FENCE) t3_wr_flags |= T3_LOCAL_FENCE_FLAG; t3_wr_opcode = T3_WR_INV_STAG; - err = iwch_build_inv_stag(wqe, wr, &t3_wr_flit_cnt); + err = build_inv_stag(wqe, wr, &t3_wr_flit_cnt); break; default: PDBG("%s post of type=%d TBD!\n", __func__, @@ -394,18 +478,24 @@ int iwch_post_receive(struct ib_qp *ibqp, struct ib_recv_wr *wr, return -EINVAL; } while (wr) { + if (wr->num_sge > T3_MAX_SGE) { + err = -EINVAL; + *bad_wr = wr; + break; + } idx = Q_PTR2IDX(qhp->wq.wptr, qhp->wq.size_log2); wqe = (union t3_wr *) (qhp->wq.queue + idx); if (num_wrs) - err = iwch_build_rdma_recv(qhp->rhp, wqe, wr); + if (wr->sg_list[0].lkey) + err = build_rdma_recv(qhp, wqe, wr); + else + err = build_zero_stag_recv(qhp, wqe, wr); else err = -ENOMEM; if (err) { *bad_wr = wr; break; } - qhp->wq.rq[Q_PTR2IDX(qhp->wq.rq_wptr, qhp->wq.rq_size_log2)] = - wr->wr_id; build_fw_riwrh((void *) wqe, T3_WR_RCV, T3_COMPLETION_FLAG, Q_GENBIT(qhp->wq.wptr, qhp->wq.size_log2), 0, sizeof(struct t3_receive_wr) >> 3, T3_SOPEOP); @@ -806,7 +896,8 @@ static int rdma_init(struct iwch_dev *rhp, struct iwch_qp *qhp, init_attr.qp_dma_size = (1UL << qhp->wq.size_log2); init_attr.rqe_count = iwch_rqes_posted(qhp); init_attr.flags = qhp->attr.mpa_attr.initiator ? MPA_INITIATOR : 0; - init_attr.flags |= capable(CAP_NET_BIND_SERVICE) ? PRIV_QP : 0; + if (!qhp->ibqp.uobject) + init_attr.flags |= PRIV_QP; if (peer2peer) { init_attr.rtr_type = RTR_READ; if (init_attr.ord == 0 && qhp->attr.mpa_attr.initiator) diff --git a/drivers/net/cxgb3/version.h b/drivers/net/cxgb3/version.h index a0177fc..29db711 100644 --- a/drivers/net/cxgb3/version.h +++ b/drivers/net/cxgb3/version.h @@ -38,7 +38,7 @@ #define DRV_VERSION "1.0-ko" /* Firmware version */ -#define FW_VERSION_MAJOR 6 +#define FW_VERSION_MAJOR 7 #define FW_VERSION_MINOR 0 #define FW_VERSION_MICRO 0 #endif /* __CHELSIO_VERSION_H */ From swise at opengridcomputing.com Tue Jul 1 13:20:14 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 15:20:14 -0500 Subject: [ofa-general] [PATCH RFC] RDMA/cxgb3: Support read-with-inv-local. Message-ID: <20080701202014.21018.1816.stgit@dell3.ogc.int> From: Steve Wise Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/cxio_wr.h | 3 ++- drivers/infiniband/hw/cxgb3/iwch_qp.c | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/cxio_wr.h b/drivers/infiniband/hw/cxgb3/cxio_wr.h index 4a8b7fb..04618f7 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_wr.h +++ b/drivers/infiniband/hw/cxgb3/cxio_wr.h @@ -242,7 +242,8 @@ struct t3_rdma_read_wr { struct fw_riwrh wrh; /* 0 */ union t3_wrid wrid; /* 1 */ u8 rdmaop; /* 2 */ - u8 reserved[3]; + u8 local_inv; + u8 reserved[2]; __be32 rem_stag; __be64 rem_to; /* 3 */ __be32 local_stag; /* 4 */ diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c index 30085c7..0526a6c 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_qp.c +++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c @@ -129,9 +129,10 @@ static int build_rdma_read(union t3_wr *wqe, struct ib_send_wr *wr, if (wr->num_sge > 1) return -EINVAL; wqe->read.rdmaop = T3_READ_REQ; + if (wr->opcode == IB_WR_RDMA_READ_WITH_INV) + wqe->read.local_inv = 1; wqe->read.reserved[0] = 0; wqe->read.reserved[1] = 0; - wqe->read.reserved[2] = 0; wqe->read.rem_stag = cpu_to_be32(wr->wr.rdma.rkey); wqe->read.rem_to = cpu_to_be64(wr->wr.rdma.remote_addr); wqe->read.local_stag = cpu_to_be32(wr->sg_list[0].lkey); @@ -401,6 +402,7 @@ int iwch_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, err = build_rdma_write(wqe, wr, &t3_wr_flit_cnt); break; case IB_WR_RDMA_READ: + case IB_WR_RDMA_READ_WITH_INV: t3_wr_opcode = T3_WR_READ; t3_wr_flags = 0; /* T3 reads are always signaled */ err = build_rdma_read(wqe, wr, &t3_wr_flit_cnt); From rdreier at cisco.com Tue Jul 1 13:25:42 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 13:25:42 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/cxgb3: Support read-with-inv-local. In-Reply-To: <20080701202014.21018.1816.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 01 Jul 2008 15:20:14 -0500") References: <20080701202014.21018.1816.stgit@dell3.ogc.int> Message-ID: This should roll up into the mem-mgt-ext support I think... does that make sense to you? - R. From swise at opengridcomputing.com Tue Jul 1 13:28:58 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 15:28:58 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/cxgb3: Support read-with-inv-local. In-Reply-To: References: <20080701202014.21018.1816.stgit@dell3.ogc.int> Message-ID: <486A938A.8020208@opengridcomputing.com> Roland Dreier wrote: > This should roll up into the mem-mgt-ext support I think... does that > make sense to you? > > - R. > Sure. From rdreier at cisco.com Tue Jul 1 13:39:29 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 13:39:29 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/cxgb3: Support read-with-inv-local. In-Reply-To: <20080701202014.21018.1816.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 01 Jul 2008 15:20:14 -0500") References: <20080701202014.21018.1816.stgit@dell3.ogc.int> Message-ID: OK, I rolled this up into the original mem mgt ext patch for cxgb3. From rdreier at cisco.com Tue Jul 1 13:57:14 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 13:57:14 -0700 Subject: [ofa-general] Re: [PATCH V3] IB/ipoib: refresh paths instead of fushing them on SM change event In-Reply-To: <486A419B.9010904@Voltaire.COM> (Moni Shoua's message of "Tue, 01 Jul 2008 17:39:23 +0300") References: <484F8A20.9070101@Voltaire.COM> <486A419B.9010904@Voltaire.COM> Message-ID: thanks, applied. a couple of notes: > In most cases, SM failover doesn't cause LID change so traffic won't stop. In > the rare cases of LID change, the remote host (the one that hadn't changed > its LID) will lose connectivity until paths are refreshed. This is no worse than > the current state. In fact, preventing the device from going down saves packets > that otherwise would be lost. is it really true that this is *always* no worse? there is the case where LIDs *did* change on an SM change event, and this patch waits for an ARP probe before discovering the LID change, while the old code immediately would look up the path again. But that seems like a less common case, and I do believe that this patch makes things better in the more common cases. I also changed the path_rec_completion() handling of old ahs, based on your code. I think the below is a little simpler, let me know if it still looks OK to you: @@ -393,6 +410,7 @@ static void path_rec_completion(int status, struct net_device *dev = path->dev; struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_ah *ah = NULL; + struct ipoib_ah *old_ah; struct ipoib_neigh *neigh, *tn; struct sk_buff_head skqueue; struct sk_buff *skb; @@ -416,6 +434,7 @@ static void path_rec_completion(int status, spin_lock_irqsave(&priv->lock, flags); + old_ah = path->ah; path->ah = ah; if (ah) { @@ -428,6 +447,17 @@ static void path_rec_completion(int status, __skb_queue_tail(&skqueue, skb); list_for_each_entry_safe(neigh, tn, &path->neigh_list, list) { + if (neigh->ah) { + WARN_ON(neigh->ah != old_ah); + /* + * Dropping the ah reference inside + * priv->lock is safe here, because we + * will hold one more reference from + * the original value of path->ah (ie + * old_ah). + */ + ipoib_put_ah(neigh->ah); + } kref_get(&path->ah->ref); neigh->ah = path->ah; memcpy(&neigh->dgid.raw, &path->pathrec.dgid.raw, @@ -450,6 +480,7 @@ static void path_rec_completion(int status, while ((skb = __skb_dequeue(&neigh->queue))) __skb_queue_tail(&skqueue, skb); } + path->valid = 1; } path->query = NULL; @@ -457,6 +488,9 @@ static void path_rec_completion(int status, spin_unlock_irqrestore(&priv->lock, flags); + if (old_ah) + ipoib_put_ah(old_ah); + while ((skb = __skb_dequeue(&skqueue))) { skb->dev = dev; if (dev_queue_xmit(skb)) From swise at opengridcomputing.com Tue Jul 1 15:33:28 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 01 Jul 2008 17:33:28 -0500 Subject: [ofa-general] [PATCH for-2.6.27] RDMA/cxgb3: Propagate HW Page Size Capabilities Message-ID: <20080701223328.15456.38444.stgit@dell3.ogc.int> From: Jon Mason cxgb3 does not currently report the page size capabilities, and incorrectly reports them internally. This version changes the bit-shifting to a static value (per Steve's request). Signed-Off-By: Jon Mason Acked-by: Steve Wise --- drivers/infiniband/hw/cxgb3/cxio_hal.h | 1 + drivers/infiniband/hw/cxgb3/iwch.c | 2 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 1 + 3 files changed, 3 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.h b/drivers/infiniband/hw/cxgb3/cxio_hal.h index a9ff32c..656fe47 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_hal.h +++ b/drivers/infiniband/hw/cxgb3/cxio_hal.h @@ -55,6 +55,7 @@ #define T3_MAX_CQ_DEPTH 8192 #define T3_MAX_NUM_STAG (1<<15) #define T3_MAX_MR_SIZE 0x100000000ULL +#define T3_PAGESIZE_MASK 0xffff000 /* 4KB-128MB */ #define T3_STAG_UNSET 0xffffffff diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c index 0608625..4489c89 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.c +++ b/drivers/infiniband/hw/cxgb3/iwch.c @@ -80,7 +80,7 @@ static void rnic_init(struct iwch_dev *rnicp) rnicp->attr.max_mem_regs = cxio_num_stags(&rnicp->rdev); rnicp->attr.max_phys_buf_entries = T3_MAX_PBL_SIZE; rnicp->attr.max_pds = T3_MAX_NUM_PD - 1; - rnicp->attr.mem_pgsizes_bitmask = 0x7FFF; /* 4KB-128MB */ + rnicp->attr.mem_pgsizes_bitmask = T3_PAGESIZE_MASK; rnicp->attr.max_mr_size = T3_MAX_MR_SIZE; rnicp->attr.can_resize_wq = 0; rnicp->attr.max_rdma_reads_per_qp = 8; diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index 4ee33e7..c6f103c 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1125,6 +1125,7 @@ static int iwch_query_device(struct ib_device *ibdev, props->hw_ver = dev->rdev.t3cdev_p->type; props->fw_ver = fw_vers_string_to_u64(dev); props->device_cap_flags = dev->device_cap_flags; + props->page_size_cap = dev->attr.mem_pgsizes_bitmask; props->vendor_id = (u32)dev->rdev.rnic_info.pdev->vendor; props->vendor_part_id = (u32)dev->rdev.rnic_info.pdev->device; props->max_mr_size = dev->attr.max_mr_size; From weiny2 at llnl.gov Tue Jul 1 18:33:05 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 1 Jul 2008 18:33:05 -0700 Subject: [ofa-general] [PATCH] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop Message-ID: <20080701183305.586a6146.weiny2@llnl.gov> >From 7a9131f07ffa29e509004fb9835a82eca93d3fac Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 1 Jul 2008 14:31:00 -0700 Subject: [PATCH] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop Signed-off-by: Ira K. Weiny --- infiniband-diags/src/saquery.c | 97 +++++++++++++++------------------------- 1 files changed, 36 insertions(+), 61 deletions(-) diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 1594cad..922703f 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -237,32 +237,11 @@ static void dump_path_record(void *data) ); } -/** - * str must be longer than 32 to hold the full gid. - * len will be checked to ensure this. - */ -static char * -sprint_gid(ib_gid_t *gid, char *str, size_t len) -{ - int i = 0; - char tmp[16]; - - assert(str != NULL); - assert(len > 32); - - str[0] = '\0'; - for (i = 0; i < 16; i++) { - sprintf(tmp, "%02X", gid->raw[i]); - strcat(str, tmp); - } - - return (str); -} - static void dump_class_port_info(void *data) { size_t GID_STR_LEN = 256; char gid_str[GID_STR_LEN]; + char gid_str2[GID_STR_LEN]; ib_class_port_info_t *class_port_info = data; printf("SA ClassPortInfo:\n" @@ -271,13 +250,13 @@ static void dump_class_port_info(void *data) "\t\tCapability mask..........0x%04X\n" "\t\tCapability mask 2........0x%08X\n" "\t\tResponse time value......0x%02X\n" - "\t\tRedirect GID.............0x%s\n" + "\t\tRedirect GID.............%s\n" "\t\tRedirect TC/SL/FL........0x%08X\n" "\t\tRedirect LID.............0x%04X\n" "\t\tRedirect PKey............0x%04X\n" "\t\tRedirect QP..............0x%08X\n" "\t\tRedirect QKey............0x%08X\n" - "\t\tTrap GID.................0x%s\n" + "\t\tTrap GID.................%s\n" "\t\tTrap TC/SL/FL............0x%08X\n" "\t\tTrap LID.................0x%04X\n" "\t\tTrap PKey................0x%04X\n" @@ -289,13 +268,13 @@ static void dump_class_port_info(void *data) cl_ntoh16(class_port_info->cap_mask), ib_class_cap_mask2(class_port_info), ib_class_resp_time_val(class_port_info), - sprint_gid(&(class_port_info->redir_gid), gid_str, GID_STR_LEN), + inet_ntop(AF_INET6, &(class_port_info->redir_gid), gid_str, GID_STR_LEN), cl_ntoh32(class_port_info->redir_tc_sl_fl), cl_ntoh16(class_port_info->redir_lid), cl_ntoh16(class_port_info->redir_pkey), cl_ntoh32(class_port_info->redir_qp), cl_ntoh32(class_port_info->redir_qkey), - sprint_gid(&(class_port_info->trap_gid), gid_str, GID_STR_LEN), + inet_ntop(AF_INET6, &(class_port_info->trap_gid), gid_str2, GID_STR_LEN), cl_ntoh32(class_port_info->trap_tc_sl_fl), cl_ntoh16(class_port_info->trap_lid), cl_ntoh16(class_port_info->trap_pkey), @@ -326,20 +305,20 @@ static void dump_portinfo_record(void *data) static void dump_multicast_group_record(void *data) { + size_t GID_STR_LEN = 256; + char gid_str[GID_STR_LEN]; ib_member_rec_t *p_mcmr = data; uint8_t sl; ib_member_get_sl_flow_hop(p_mcmr->sl_flow_hop, &sl, NULL, NULL); printf("MCMemberRecord group dump:\n" - "\t\tMGID....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tMGID....................%s\n" "\t\tMlid....................0x%X\n" "\t\tMtu.....................0x%X\n" "\t\tpkey....................0x%X\n" "\t\tRate....................0x%X\n" "\t\tSL......................0x%X\n" "", - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, GID_STR_LEN), cl_ntoh16( p_mcmr->mlid ), p_mcmr->mtu, cl_ntoh16( p_mcmr->pkey ), @@ -350,8 +329,10 @@ static void dump_multicast_group_record(void *data) static void dump_multicast_member_record(void *data) { + size_t GID_STR_LEN = 256; + char gid_str[GID_STR_LEN]; + char gid_str2[GID_STR_LEN]; ib_member_rec_t *p_mcmr = data; - uint64_t gid_prefix = cl_ntoh64( p_mcmr->port_gid.unicast.prefix ); uint64_t gid_interface_id = cl_ntoh64( p_mcmr->port_gid.unicast.interface_id ); uint16_t mlid = cl_ntoh16( p_mcmr->mlid ); int i = 0; @@ -371,29 +352,26 @@ static void dump_multicast_member_record(void *data) if (requested_name) { if (strtol(requested_name, NULL, 0) == mlid) { - printf("\t\tPortGid.................0x%016" PRIx64 " : " - "0x%016" PRIx64 " (%s)\n", - gid_prefix, - gid_interface_id, + printf("\t\tPortGid.................%s (%s)\n", + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, + gid_str, GID_STR_LEN), node_name ); } } else { printf("MCMemberRecord member dump:\n" - "\t\tMGID....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tMGID....................%s\n" "\t\tMlid....................0x%X\n" - "\t\tPortGid.................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tPortGid.................%s\n" "\t\tScopeState..............0x%X\n" "\t\tProxyJoin...............0x%X\n" "\t\tNodeDescription.........%s\n" "", - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, + GID_STR_LEN), cl_ntoh16( p_mcmr->mlid ), - gid_prefix, - gid_interface_id, + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, + gid_str2, GID_STR_LEN), p_mcmr->scope_state, p_mcmr->proxy_join, node_name @@ -403,6 +381,8 @@ static void dump_multicast_member_record(void *data) static void dump_service_record(void *data) { + size_t GID_STR_LEN = 256; + char gid_str[GID_STR_LEN]; char buf_service_key[35]; char buf_service_name[65]; ib_service_record_t *p_sr = data; @@ -430,8 +410,7 @@ static void dump_service_record(void *data) printf("ServiceRecord dump:\n" "\t\tServiceID...............0x%016" PRIx64 "\n" - "\t\tServiceGID..............0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tServiceGID..............%s\n" "\t\tServiceP_Key............0x%X\n" "\t\tServiceLease............0x%X\n" "\t\tServiceKey..............%s\n" @@ -468,8 +447,7 @@ static void dump_service_record(void *data) "\t\tServiceData64.2.........0x%016" PRIx64 "\n" "", cl_ntoh64( p_sr->service_id ), - cl_ntoh64( p_sr->service_gid.unicast.prefix ), - cl_ntoh64( p_sr->service_gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_sr->service_gid.raw, gid_str, GID_STR_LEN), cl_ntoh16( p_sr->service_pkey ), cl_ntoh32( p_sr->service_lease ), buf_service_key, @@ -501,6 +479,9 @@ static void dump_service_record(void *data) static void dump_inform_info_record(void *data) { + size_t GID_STR_LEN = 256; + char gid_str[GID_STR_LEN]; + char gid_str2[GID_STR_LEN]; ib_inform_info_record_t *p_iir = data; uint32_t qpn; uint8_t resp_time_val; @@ -510,11 +491,10 @@ static void dump_inform_info_record(void *data) if (p_iir->inform_info.is_generic) { printf("InformInfoRecord dump:\n" "\t\tRID\n" - "\t\tSubscriberGID...........0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tSubscriberGID...........%s\n" "\t\tSubscriberEnum..........0x%X\n" "\t\tInformInfo dump:\n" - "\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" + "\t\tgid.....................%s\n" "\t\tlid_range_begin.........0x%X\n" "\t\tlid_range_end...........0x%X\n" "\t\tis_generic..............0x%X\n" @@ -525,11 +505,9 @@ static void dump_inform_info_record(void *data) "\t\tresp_time_val...........0x%X\n" "\t\tnode_type...............0x%06X\n" "", - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, gid_str, GID_STR_LEN), cl_ntoh16( p_iir->subscriber_enum ), - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, gid_str2, GID_STR_LEN), cl_ntoh16( p_iir->inform_info.lid_range_begin ), cl_ntoh16( p_iir->inform_info.lid_range_end ), p_iir->inform_info.is_generic, @@ -543,11 +521,10 @@ static void dump_inform_info_record(void *data) } else { printf("InformInfoRecord dump:\n" "\t\tRID\n" - "\t\tSubscriberGID...........0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tSubscriberGID...........%s\n" "\t\tSubscriberEnum..........0x%X\n" "\t\tInformInfo dump:\n" - "\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" + "\t\tgid.....................%s\n" "\t\tlid_range_begin.........0x%X\n" "\t\tlid_range_end...........0x%X\n" "\t\tis_generic..............0x%X\n" @@ -558,11 +535,9 @@ static void dump_inform_info_record(void *data) "\t\tresp_time_val...........0x%X\n" "\t\tvendor_id...............0x%06X\n" "", - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, gid_str, GID_STR_LEN), cl_ntoh16( p_iir->subscriber_enum ), - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, gid_str2, GID_STR_LEN), cl_ntoh16( p_iir->inform_info.lid_range_begin ), cl_ntoh16( p_iir->inform_info.lid_range_end ), p_iir->inform_info.is_generic, -- 1.5.4.5 From weiny2 at llnl.gov Tue Jul 1 18:33:24 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 1 Jul 2008 18:33:24 -0700 Subject: [ofa-general] [PATCH] infiniband-diags/src/ibaddr.c: convert GID prints to use inet_ntop Message-ID: <20080701183324.7f2e2e71.weiny2@llnl.gov> >From c6bd45d3512809957e6551e1ff15766822d63679 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 1 Jul 2008 15:48:54 -0700 Subject: [PATCH] infiniband-diags/src/ibaddr.c: convert GID prints to use inet_ntop Signed-off-by: Ira K. Weiny --- infiniband-diags/src/ibaddr.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c index 6c56a3e..7362640 100644 --- a/infiniband-diags/src/ibaddr.c +++ b/infiniband-diags/src/ibaddr.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include @@ -52,6 +53,8 @@ char *argv0 = "ibaddr"; static int ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) { + size_t GID_STR_LEN = 256; + char gid_str[GID_STR_LEN]; uint8_t portinfo[64]; uint8_t nodeinfo[64]; uint64_t guid, prefix; @@ -74,7 +77,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) mad_encode_field(gid, IB_GID_GUID_F, &guid); if (show_gid) { - printf("GID 0x%s%s ", mad_dump_val(IB_GID_PREFIX_F, buf1, sizeof buf1, &prefix), mad_dump_val(IB_GID_GUID_F, buf2, sizeof buf2, &guid)); + printf("GID %s ", inet_ntop(AF_INET6, gid, gid_str, GID_STR_LEN)); } if (show_lid > 0) -- 1.5.4.5 From rdreier at cisco.com Tue Jul 1 21:07:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 21:07:05 -0700 Subject: [ofa-general] [PATCH] RDMA/nes: Encapsulate logic nes_put_cqp_request() Message-ID: The iw_nes driver repeats the logic if (atomic_dec_and_test(&cqp_request->refcount)) { if (cqp_request->dynamic) { kfree(cqp_request); } else { spin_lock_irqsave(&nesdev->cqp.lock, flags); list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); spin_unlock_irqrestore(&nesdev->cqp.lock, flags); } } over and over. Wrap this up in functions nes_free_cqp_request() and nes_put_cqp_request() to simplify such code. In addition to making the source smaller and more readable, this shrinks the compiled code quite a bit: add/remove: 2/0 grow/shrink: 0/13 up/down: 164/-1692 (-1528) function old new delta nes_free_cqp_request - 147 +147 nes_put_cqp_request - 17 +17 nes_modify_qp 2316 2293 -23 nes_hw_modify_qp 737 657 -80 nes_dereg_mr 945 860 -85 flush_wqes 501 416 -85 nes_manage_apbvt 648 560 -88 nes_reg_mr 1117 1026 -91 nes_cqp_ce_handler 927 769 -158 nes_alloc_mw 1052 884 -168 nes_create_qp 5314 5141 -173 nes_alloc_fmr 2212 2035 -177 nes_destroy_cq 1097 918 -179 nes_create_cq 2787 2598 -189 nes_dealloc_mw 762 566 -196 Signed-off-by: Roland Dreier --- Here's a cleanup I'll queue for 2.6.27 if it's OK with you. drivers/infiniband/hw/nes/nes.h | 4 + drivers/infiniband/hw/nes/nes_hw.c | 60 ++--------- drivers/infiniband/hw/nes/nes_utils.c | 24 ++++ drivers/infiniband/hw/nes/nes_verbs.c | 189 +++++--------------------------- 4 files changed, 66 insertions(+), 211 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes.h b/drivers/infiniband/hw/nes/nes.h index 61b46e9..fe88bec 100644 --- a/drivers/infiniband/hw/nes/nes.h +++ b/drivers/infiniband/hw/nes/nes.h @@ -538,6 +538,10 @@ void nes_read_1G_phy_reg(struct nes_device *, u8, u8, u16 *); void nes_write_10G_phy_reg(struct nes_device *, u16, u8, u16, u16); void nes_read_10G_phy_reg(struct nes_device *, u8, u8, u16); struct nes_cqp_request *nes_get_cqp_request(struct nes_device *); +void nes_free_cqp_request(struct nes_device *nesdev, + struct nes_cqp_request *cqp_request); +void nes_put_cqp_request(struct nes_device *nesdev, + struct nes_cqp_request *cqp_request); void nes_post_cqp_request(struct nes_device *, struct nes_cqp_request *, int); int nes_arp_table(struct nes_device *, u32, u8 *, u32); void nes_mh_fix(unsigned long); diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c index d3278f1..80e4866 100644 --- a/drivers/infiniband/hw/nes/nes_hw.c +++ b/drivers/infiniband/hw/nes/nes_hw.c @@ -2710,39 +2710,11 @@ static void nes_cqp_ce_handler(struct nes_device *nesdev, struct nes_hw_cq *cq) barrier(); cqp_request->request_done = 1; wake_up(&cqp_request->waitq); - if (atomic_dec_and_test(&cqp_request->refcount)) { - nes_debug(NES_DBG_CQP, "CQP request %p (opcode 0x%02X) freed.\n", - cqp_request, - le32_to_cpu(cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX])&0x3f); - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } - } else if (cqp_request->callback) { - /* Envoke the callback routine */ - cqp_request->cqp_callback(nesdev, cqp_request); - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } + nes_put_cqp_request(nesdev, cqp_request); } else { - nes_debug(NES_DBG_CQP, "CQP request %p (opcode 0x%02X) freed.\n", - cqp_request, - le32_to_cpu(cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX]) & 0x3f); - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } + if (cqp_request->callback) + cqp_request->cqp_callback(nesdev, cqp_request); + nes_free_cqp_request(nesdev, cqp_request); } } else { wake_up(&nesdev->cqp.waitq); @@ -3149,7 +3121,6 @@ int nes_manage_apbvt(struct nes_vnic *nesvnic, u32 accel_local_port, { struct nes_device *nesdev = nesvnic->nesdev; struct nes_hw_cqp_wqe *cqp_wqe; - unsigned long flags; struct nes_cqp_request *cqp_request; int ret = 0; u16 major_code; @@ -3184,15 +3155,9 @@ int nes_manage_apbvt(struct nes_vnic *nesvnic, u32 accel_local_port, nes_debug(NES_DBG_QP, "Completed, ret=%u, CQP Major:Minor codes = 0x%04X:0x%04X\n", ret, cqp_request->major_code, cqp_request->minor_code); major_code = cqp_request->major_code; - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + + nes_put_cqp_request(nesdev, cqp_request); + if (!ret) return -ETIME; else if (major_code) @@ -3262,7 +3227,6 @@ void nes_manage_arp_cache(struct net_device *netdev, unsigned char *mac_addr, void flush_wqes(struct nes_device *nesdev, struct nes_qp *nesqp, u32 which_wq, u32 wait_completion) { - unsigned long flags; struct nes_cqp_request *cqp_request; struct nes_hw_cqp_wqe *cqp_wqe; int ret; @@ -3294,14 +3258,6 @@ void flush_wqes(struct nes_device *nesdev, struct nes_qp *nesqp, nes_debug(NES_DBG_QP, "Flush SQ QP WQEs completed, ret=%u," " CQP Major:Minor codes = 0x%04X:0x%04X\n", ret, cqp_request->major_code, cqp_request->minor_code); - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + nes_put_cqp_request(nesdev, cqp_request); } } diff --git a/drivers/infiniband/hw/nes/nes_utils.c b/drivers/infiniband/hw/nes/nes_utils.c index fe83d1b..82abe9b 100644 --- a/drivers/infiniband/hw/nes/nes_utils.c +++ b/drivers/infiniband/hw/nes/nes_utils.c @@ -567,6 +567,30 @@ struct nes_cqp_request *nes_get_cqp_request(struct nes_device *nesdev) return cqp_request; } +void nes_free_cqp_request(struct nes_device *nesdev, + struct nes_cqp_request *cqp_request) +{ + unsigned long flags; + + nes_debug(NES_DBG_CQP, "CQP request %p (opcode 0x%02X) freed.\n", + cqp_request, + le32_to_cpu(cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX])&0x3f); + + if (cqp_request->dynamic) { + kfree(cqp_request); + } else { + spin_lock_irqsave(&nesdev->cqp.lock, flags); + list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); + spin_unlock_irqrestore(&nesdev->cqp.lock, flags); + } +} + +void nes_put_cqp_request(struct nes_device *nesdev, + struct nes_cqp_request *cqp_request) +{ + if (atomic_dec_and_test(&cqp_request->refcount)) + nes_free_cqp_request(nesdev, cqp_request); +} /** * nes_post_cqp_request diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index d617da9..464a98a 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -55,7 +55,6 @@ static void nes_unregister_ofa_device(struct nes_ib_device *nesibdev); * nes_alloc_mw */ static struct ib_mw *nes_alloc_mw(struct ib_pd *ibpd) { - unsigned long flags; struct nes_pd *nespd = to_nespd(ibpd); struct nes_vnic *nesvnic = to_nesvnic(ibpd->device); struct nes_device *nesdev = nesvnic->nesdev; @@ -128,15 +127,7 @@ static struct ib_mw *nes_alloc_mw(struct ib_pd *ibpd) { " CQP Major:Minor codes = 0x%04X:0x%04X.\n", stag, ret, cqp_request->major_code, cqp_request->minor_code); if ((!ret) || (cqp_request->major_code)) { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + nes_put_cqp_request(nesdev, cqp_request); kfree(nesmr); nes_free_resource(nesadapter, nesadapter->allocated_mrs, stag_index); if (!ret) { @@ -144,17 +135,8 @@ static struct ib_mw *nes_alloc_mw(struct ib_pd *ibpd) { } else { return ERR_PTR(-ENOMEM); } - } else { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } } + nes_put_cqp_request(nesdev, cqp_request); nesmr->ibmw.rkey = stag; nesmr->mode = IWNES_MEMREG_TYPE_MW; @@ -178,7 +160,6 @@ static int nes_dealloc_mw(struct ib_mw *ibmw) struct nes_hw_cqp_wqe *cqp_wqe; struct nes_cqp_request *cqp_request; int err = 0; - unsigned long flags; int ret; /* Deallocate the window with the adapter */ @@ -204,32 +185,12 @@ static int nes_dealloc_mw(struct ib_mw *ibmw) nes_debug(NES_DBG_MR, "Deallocate STag completed, wait_event_timeout ret = %u," " CQP Major:Minor codes = 0x%04X:0x%04X.\n", ret, cqp_request->major_code, cqp_request->minor_code); - if ((!ret) || (cqp_request->major_code)) { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } - if (!ret) { - err = -ETIME; - } else { - err = -EIO; - } - } else { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } - } + if (!ret) + err = -ETIME; + else if (cqp_request->major_code) + err = -EIO; + + nes_put_cqp_request(nesdev, cqp_request); nes_free_resource(nesadapter, nesadapter->allocated_mrs, (ibmw->rkey & 0x0fffff00) >> 8); @@ -526,29 +487,11 @@ static struct ib_fmr *nes_alloc_fmr(struct ib_pd *ibpd, stag, ret, cqp_request->major_code, cqp_request->minor_code); if ((!ret) || (cqp_request->major_code)) { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + nes_put_cqp_request(nesdev, cqp_request); ret = (!ret) ? -ETIME : -EIO; goto failed_leaf_vpbl_pages_alloc; - } else { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } } - + nes_put_cqp_request(nesdev, cqp_request); nesfmr->nesmr.ibfmr.lkey = stag; nesfmr->nesmr.ibfmr.rkey = stag; nesfmr->attr = *ibfmr_attr; @@ -1487,15 +1430,7 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd, nesqp->hwqp.qp_id, ret, nesdev->cqp.sq_head, nesdev->cqp.sq_tail, cqp_request->major_code, cqp_request->minor_code); if ((!ret) || (cqp_request->major_code)) { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + nes_put_cqp_request(nesdev, cqp_request); nes_free_resource(nesadapter, nesadapter->allocated_qps, qp_num); nes_free_qp_mem(nesdev, nesqp,virt_wqs); kfree(nesqp->allocated_buffer); @@ -1504,18 +1439,10 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd, } else { return ERR_PTR(-EIO); } - } else { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } } + nes_put_cqp_request(nesdev, cqp_request); + if (ibpd->uobject) { uresp.mmap_sq_db_index = nesqp->mmap_sq_db_index; uresp.actual_sq_size = sq_size; @@ -1827,32 +1754,15 @@ static struct ib_cq *nes_create_cq(struct ib_device *ibdev, int entries, nes_debug(NES_DBG_CQ, "Create iWARP CQ%u completed, wait_event_timeout ret = %d.\n", nescq->hw_cq.cq_number, ret); if ((!ret) || (cqp_request->major_code)) { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + nes_put_cqp_request(nesdev, cqp_request); if (!context) pci_free_consistent(nesdev->pcidev, nescq->cq_mem_size, mem, nescq->hw_cq.cq_pbase); nes_free_resource(nesadapter, nesadapter->allocated_cqs, cq_num); kfree(nescq); return ERR_PTR(-EIO); - } else { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } } + nes_put_cqp_request(nesdev, cqp_request); if (context) { /* free the nespbl */ @@ -1942,37 +1852,18 @@ static int nes_destroy_cq(struct ib_cq *ib_cq) " CQP Major:Minor codes = 0x%04X:0x%04X.\n", nescq->hw_cq.cq_number, ret, cqp_request->major_code, cqp_request->minor_code); - if ((!ret) || (cqp_request->major_code)) { - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } - if (!ret) { - nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy timeout expired\n", + if (!ret) { + nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy timeout expired\n", nescq->hw_cq.cq_number); - ret = -ETIME; - } else { - nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy failed\n", + ret = -ETIME; + } else if (cqp_request->major_code) { + nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy failed\n", nescq->hw_cq.cq_number); - ret = -EIO; - } + ret = -EIO; } else { ret = 0; - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } } + nes_put_cqp_request(nesdev, cqp_request); if (nescq->cq_mem_size) pci_free_consistent(nesdev->pcidev, nescq->cq_mem_size, @@ -2105,15 +1996,8 @@ static int nes_reg_mr(struct nes_device *nesdev, struct nes_pd *nespd, " CQP Major:Minor codes = 0x%04X:0x%04X.\n", stag, ret, cqp_request->major_code, cqp_request->minor_code); major_code = cqp_request->major_code; - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + nes_put_cqp_request(nesdev, cqp_request); + if (!ret) return -ETIME; else if (major_code) @@ -2771,15 +2655,9 @@ static int nes_dereg_mr(struct ib_mr *ib_mr) major_code = cqp_request->major_code; minor_code = cqp_request->minor_code; - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + + nes_put_cqp_request(nesdev, cqp_request); + if (!ret) { nes_debug(NES_DBG_MR, "Timeout waiting to destroy STag," " ib_mr=%p, rkey = 0x%08X\n", @@ -2904,7 +2782,6 @@ int nes_hw_modify_qp(struct nes_device *nesdev, struct nes_qp *nesqp, /* struct iw_cm_id *cm_id = nesqp->cm_id; */ /* struct iw_cm_event cm_event; */ struct nes_cqp_request *cqp_request; - unsigned long flags; int ret; u16 major_code; @@ -2950,15 +2827,9 @@ int nes_hw_modify_qp(struct nes_device *nesdev, struct nes_qp *nesqp, nesqp->hwqp.qp_id, cqp_request->major_code, cqp_request->minor_code, next_iwarp_state); } - if (atomic_dec_and_test(&cqp_request->refcount)) { - if (cqp_request->dynamic) { - kfree(cqp_request); - } else { - spin_lock_irqsave(&nesdev->cqp.lock, flags); - list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); - } - } + + nes_put_cqp_request(nesdev, cqp_request); + if (!ret) return -ETIME; else if (major_code) -- 1.5.6 From rdreier at cisco.com Tue Jul 1 21:07:08 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 21:07:08 -0700 Subject: [ofa-general] [PATCH] RDMA/nes: Remove write-only fields of struct nes_vnic Message-ID: struct nes_vnic.get_cqp_request and .post_cqp_request are assigned but never used. Remove them. Signed-off-by: Roland Dreier --- And another one... let me know if there is some plan to use these fields in the future and I'll drop this. drivers/infiniband/hw/nes/nes_hw.c | 2 -- drivers/infiniband/hw/nes/nes_hw.h | 2 -- 2 files changed, 0 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c index 80e4866..0cea58b 100644 --- a/drivers/infiniband/hw/nes/nes_hw.c +++ b/drivers/infiniband/hw/nes/nes_hw.c @@ -1591,8 +1591,6 @@ int nes_init_nic_qp(struct nes_device *nesdev, struct net_device *netdev) cpu_to_le32((u32)((u64)nesvnic->nic.frag_paddr[counter] >> 32)); } - nesvnic->get_cqp_request = nes_get_cqp_request; - nesvnic->post_cqp_request = nes_post_cqp_request; nesvnic->mcrq_mcast_filter = NULL; spin_lock_init(&nesvnic->nic.sq_lock); diff --git a/drivers/infiniband/hw/nes/nes_hw.h b/drivers/infiniband/hw/nes/nes_hw.h index 745bf94..4314ca2 100644 --- a/drivers/infiniband/hw/nes/nes_hw.h +++ b/drivers/infiniband/hw/nes/nes_hw.h @@ -1171,8 +1171,6 @@ struct nes_vnic { struct nes_hw_nic_cq nic_cq; u32 mcrq_qp_id; struct nes_ucontext *mcrq_ucontext; - struct nes_cqp_request* (*get_cqp_request)(struct nes_device *nesdev); - void (*post_cqp_request)(struct nes_device*, struct nes_cqp_request *, int); int (*mcrq_mcast_filter)( struct nes_vnic* nesvnic, __u8* dmi_addr ); struct net_device_stats netstats; /* used to put the netdev on the adapters logical port list */ -- 1.5.6 From rdreier at cisco.com Tue Jul 1 21:09:24 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 01 Jul 2008 21:09:24 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27] RDMA/cxgb3: Propagate HW Page Size Capabilities In-Reply-To: <20080701223328.15456.38444.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 01 Jul 2008 17:33:28 -0500") References: <20080701223328.15456.38444.stgit@dell3.ogc.int> Message-ID: thanks, applied. From avi at qumranet.com Tue Jul 1 21:58:55 2008 From: avi at qumranet.com (Avi Kivity) Date: Wed, 02 Jul 2008 07:58:55 +0300 Subject: [ofa-general] Re: [PATCH 0 of 3] mmu notifier v18 for -mm In-Reply-To: <20080701214415.b3f93706.akpm@linux-foundation.org> References: <20080701214415.b3f93706.akpm@linux-foundation.org> Message-ID: <486B0B0F.5010605@qumranet.com> Andrew Morton wrote: > On Thu, 26 Jun 2008 02:26:56 +0200 Andrea Arcangeli wrote: > > >> Hello, >> >> Christoph suggested me to repost v18 for merging in -mm, to give it more >> exposure before the .27 merge window opens. There's no code change compared to >> the previous v18 submission (the only change is the correction in the comment >> in the mm_take_all_locks patch rightfully pointed out by Linus). >> >> Full patchset including other XPMEM support patches can be found here: >> >> http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.26-rc7/mmu-notifier-v18 >> >> Only the three patches of the patchset I'm submitting here by email are ready >> for merging, the rest you can find in the website is not ready for merging yet >> for various performance degradations, lots of the XPMEM patches needs to be >> elaborated to avoid any slowdown for the non-XPMEM case, but I keep >> maintaining them to make life easier to XPMEM current development and later we >> can keep work on them to make them suitable for inclusion to avoid any >> performance degradation risk. >> > > I'm a bit concerned about merging the first three patches when there > are eleven more patches of which some, afacit, are required to make > these three actually useful. Committing these three would be signing a > blank cheque. > > The first three are useful for kvm, gru, and likely drm and rdma nics. It is only xpmem which requires the other eleven patches. > Because if we hit strong objections with the later patches we end up in a > cant-go-forward, cant-go-backward situation. > > No, we end up in a some-people-are-happy, some-have-to-redo-their-homework situation. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. From monis at Voltaire.COM Tue Jul 1 22:55:23 2008 From: monis at Voltaire.COM (Moni Shoua) Date: Wed, 02 Jul 2008 08:55:23 +0300 Subject: [ofa-general] Re: [PATCH V3] IB/ipoib: refresh paths instead of fushing them on SM change event In-Reply-To: References: <484F8A20.9070101@Voltaire.COM> <486A419B.9010904@Voltaire.COM> Message-ID: <486B184B.9020107@Voltaire.COM> Roland Dreier wrote: > thanks, applied. a couple of notes: > > is it really true that this is *always* no worse? there is the case > where LIDs *did* change on an SM change event, and this patch waits for > an ARP probe before discovering the LID change, while the old code > immediately would look up the path again. But that seems like a less > common case, and I do believe that this patch makes things better in the > more common cases. You are right, the "not worse" has a limited warranty. The rare case you described is the exception. However, I already have a way to avoid waiting to an ARP probe by queuing a task of refresh to a work_queue instead of marking the path invalid. I hope to post a patch that does it soon. > > I also changed the path_rec_completion() handling of old ahs, based on > your code. I think the below is a little simpler, let me know if it > still looks OK to you: Looks OK to me. thanks MoniS > > @@ -393,6 +410,7 @@ static void path_rec_completion(int status, > struct net_device *dev = path->dev; > struct ipoib_dev_priv *priv = netdev_priv(dev); > struct ipoib_ah *ah = NULL; > + struct ipoib_ah *old_ah; > struct ipoib_neigh *neigh, *tn; > struct sk_buff_head skqueue; > struct sk_buff *skb; > @@ -416,6 +434,7 @@ static void path_rec_completion(int status, > > spin_lock_irqsave(&priv->lock, flags); > > + old_ah = path->ah; > path->ah = ah; > > if (ah) { > @@ -428,6 +447,17 @@ static void path_rec_completion(int status, > __skb_queue_tail(&skqueue, skb); > > list_for_each_entry_safe(neigh, tn, &path->neigh_list, list) { > + if (neigh->ah) { > + WARN_ON(neigh->ah != old_ah); > + /* > + * Dropping the ah reference inside > + * priv->lock is safe here, because we > + * will hold one more reference from > + * the original value of path->ah (ie > + * old_ah). > + */ > + ipoib_put_ah(neigh->ah); > + } > kref_get(&path->ah->ref); > neigh->ah = path->ah; > memcpy(&neigh->dgid.raw, &path->pathrec.dgid.raw, > @@ -450,6 +480,7 @@ static void path_rec_completion(int status, > while ((skb = __skb_dequeue(&neigh->queue))) > __skb_queue_tail(&skqueue, skb); > } > + path->valid = 1; > } > > path->query = NULL; > @@ -457,6 +488,9 @@ static void path_rec_completion(int status, > > spin_unlock_irqrestore(&priv->lock, flags); > > + if (old_ah) > + ipoib_put_ah(old_ah); > + > while ((skb = __skb_dequeue(&skqueue))) { > skb->dev = dev; > if (dev_queue_xmit(skb)) > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From ogerlitz at voltaire.com Wed Jul 2 00:33:41 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 02 Jul 2008 10:33:41 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <20080701134229.GC14384@mtls03> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> <20080701071559.GE6024@mtls03> <486A1CB1.1020904@voltaire.com> <20080701134229.GC14384@mtls03> Message-ID: <486B2F55.5020704@voltaire.com> Eli Cohen wrote: > The fact that I did not hit such an assert does not mean I may avoid taking the rtnl lock but it appears to me that the issue is not well undrestood. Yes, linux networking internals are not easy to get... I suggest that you send netdev an email with more concrete subject - eg "required rtnl locking for netdev event notifiers..." which has larger chance to catch the eye of the maintainers. Or. From eli at dev.mellanox.co.il Wed Jul 2 01:08:55 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 2 Jul 2008 11:08:55 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <486B2F55.5020704@voltaire.com> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> <20080701071559.GE6024@mtls03> <486A1CB1.1020904@voltaire.com> <20080701134229.GC14384@mtls03> <486B2F55.5020704@voltaire.com> Message-ID: <20080702080855.GF14384@mtls03> On Wed, Jul 02, 2008 at 10:33:41AM +0300, Or Gerlitz wrote: > Yes, linux networking internals are not easy to get... I suggest that you > send netdev an email with more concrete subject - eg "required rtnl locking > for netdev event notifiers..." which has larger chance to catch the eye of > the maintainers. > That's what I was thinking about too. Yes, I'll send an email. From eli at dev.mellanox.co.il Wed Jul 2 01:54:09 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 2 Jul 2008 11:54:09 +0300 Subject: [ofa-general] locking requirements when calling dev_set_mtu() Message-ID: <20080702085409.GA13098@mtls03> Hi, I have the need to change the MTU of IP over IB from the driver code and not from the shell. Looking for the right way to do it I came across the dev_set_mtu() which seems appropriate as it handles all notifications too. However, it is not clear to me whether I have to wrap the call to dev_set_mtu() with any locks. Specifically, I was referred to a mail thread that suggests that I have to use rtnl_lock()/rtnl_unlock(): http://marc.info/?l=linux-netdev&m=121201324611292&w=2 With reference to the above URL, I would say that it is possible for me to hit to the assertion ASSERT_RTNL() if I don't take rtnl_lock prior to calling dev_set_mtu(). However I don't hit it, though I can't count on that there will not be a case that any function registered for notification would expect to see the lock acquired. Can someone shed light on this? Thanks, Eli From vlad at lists.openfabrics.org Wed Jul 2 02:42:01 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Wed, 2 Jul 2008 02:42:01 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080702-0200 daily build status Message-ID: <20080702094202.7EFE8E60C49@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c: At top level: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: error: unknown field 'change_queue_depth' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080702-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From jackm at dev.mellanox.co.il Wed Jul 2 04:59:21 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 2 Jul 2008 14:59:21 +0300 Subject: [ofa-general] Re: [PATCH 6/11] core: XRC - file descriptors associated with XRC domains In-Reply-To: References: <200806231601.38023.jackm@dev.mellanox.co.il> Message-ID: <200807021459.21547.jackm@dev.mellanox.co.il> On Tuesday 24 June 2008 00:08, Roland Dreier wrote: > > +static struct inode * xrc_fd2inode(unsigned int fd) > > +{ > > + struct file * f = fget(fd); > > + > > + if (!f) > > + return NULL; > > + > > + return f->f_dentry->d_inode; > > +} > > Am I missing something, or is there no fput() that matches this fget()? You're right. I'm adding this after I grab the inode and in the error path in procedure ib_uverbs_open_xrc_domain(). > Is there some reason why we don't need an igrab() of the inode we're > using here? (If we do need it then a corresponding iput() when we're > done with the inode is required of course) We do need igrab/iput. The algorithm here is to use the fd to get to the inode, and then grab the inode (to guarantee that it is not deleted out from under). Once I have grabbed the inode, I don't need the fd anymore (so that I can fput()). I am using igrab/iput in the patch (easy to miss): +static int insert_xrcd(struct ib_device *dev, struct inode *i_n, + struct ib_xrcd *xrcd) +{ + int ret; + + ret = xrcd_table_insert(dev, i_n, xrcd); + if (!ret) + igrab(i_n); + + return ret; +} ... +static void xrcd_table_delete(struct ib_device *dev, + struct inode *i_n) +{ + struct xrcd_table_entry *entry = xrcd_table_search(dev, i_n); + + if (entry) { + iput(i_n); + rb_erase(&entry->node, &dev->ib_uverbs_xrcd_table); + kfree(entry); + } +} At second look, its still a bit messy. I can get rid of insert_xrcd entirely, and just move the igrab to xrcd_table_insert. (as is done for xrcd_table_delete/iput). - Jack From pradeep at us.ibm.com Wed Jul 2 06:10:13 2008 From: pradeep at us.ibm.com (Pradeep Satyanarayana) Date: Wed, 2 Jul 2008 06:10:13 -0700 Subject: [ofa-general] Re: [PATCH] - [resend] Corrects a race in ipoib_cm_post_receive_nonsrq() In-Reply-To: Message-ID: Roland, Thanks for writing up this patch. I have been extremely busy dealing with some hot issues and hence the delayed response. I did see Shirley's response and yes it looks okay to me too. In the same vein that we have been distinguishing the nonsrq functions (from the generic or srq ones), would you consider renaming ipoib_cm_init_rx_wr() to something with nonsrq suffix? Thanks! Pradeep pradeep at us.ibm.com Roland Dreier 06/27/2008 03:26 PM To dwilder at linux.vnet.ibm.com, Pradeep Satyanarayana/Beaverton/IBM at IBMUS cc general at lists.openfabrics.org, Shirley Ma/Beaverton/IBM at IBMUS Subject Re: [PATCH] - [resend] Corrects a race in ipoib_cm_post_receive_nonsrq() I just added this patch to my tree, which I hope should fix this problem without adding a huge stack space allocation (which might cause other issues under load later). It's a little more complicated but I feel better about fixing things this way. Can you guys review/test this and let me know if it looks OK? Thanks, Roland >From 93c47a64b0bf01b69e711698c3bc1b10deb1ca6f Mon Sep 17 00:00:00 2001 From: Roland Dreier Date: Fri, 27 Jun 2008 15:24:15 -0700 Subject: [PATCH] IPoIB/cm: Fix racy use of receive WR/SGL in ipoib_cm_post_receive_nonsrq() For devices that don't support SRQs, ipoib_cm_post_receive_nonsrq() is called from both ipoib_cm_handle_rx_wc() and ipoib_cm_nonsrq_init_rx(), and these two callers are not synchronized against each other. However, ipoib_cm_post_receive_nonsrq() always reuses the same receive work request and scatter list structures, so multiple callers can end up stepping on each other, which leads to posting garbled work requests. Fix this by having the caller pass in the ib_recv_wr and ib_sge structures to use, and allocating new local structures in ipoib_cm_nonsrq_init_rx(). Based on a patch by Pradeep Satyanarayana and David Wilder , with debugging help from Hoang-Nam Nguyen . Signed-off-by: Roland Dreier --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 63 +++++++++++++++++++++++-------- 1 files changed, 47 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 6223fc3..37bf67b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -111,18 +111,20 @@ static int ipoib_cm_post_receive_srq(struct net_device *dev, int id) } static int ipoib_cm_post_receive_nonsrq(struct net_device *dev, - struct ipoib_cm_rx *rx, int id) + struct ipoib_cm_rx *rx, + struct ib_recv_wr *wr, + struct ib_sge *sge, int id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_recv_wr *bad_wr; int i, ret; - priv->cm.rx_wr.wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV; + wr->wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV; for (i = 0; i < IPOIB_CM_RX_SG; ++i) - priv->cm.rx_sge[i].addr = rx->rx_ring[id].mapping[i]; + sge[i].addr = rx->rx_ring[id].mapping[i]; - ret = ib_post_recv(rx->qp, &priv->cm.rx_wr, &bad_wr); + ret = ib_post_recv(rx->qp, wr, &bad_wr); if (unlikely(ret)) { ipoib_warn(priv, "post recv failed for buf %d (%d)\n", id, ret); ipoib_cm_dma_unmap_rx(priv, IPOIB_CM_RX_SG - 1, @@ -320,10 +322,33 @@ static int ipoib_cm_modify_rx_qp(struct net_device *dev, return 0; } +static void ipoib_cm_init_rx_wr(struct net_device *dev, + struct ib_recv_wr *wr, + struct ib_sge *sge) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + for (i = 0; i < priv->cm.num_frags; ++i) + sge[i].lkey = priv->mr->lkey; + + sge[0].length = IPOIB_CM_HEAD_SIZE; + for (i = 1; i < priv->cm.num_frags; ++i) + sge[i].length = PAGE_SIZE; + + wr->next = NULL; + wr->sg_list = priv->cm.rx_sge; + wr->num_sge = priv->cm.num_frags; +} + static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_id, struct ipoib_cm_rx *rx) { struct ipoib_dev_priv *priv = netdev_priv(dev); + struct { + struct ib_recv_wr wr; + struct ib_sge sge[IPOIB_CM_RX_SG]; + } *t; int ret; int i; @@ -331,6 +356,14 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i if (!rx->rx_ring) return -ENOMEM; + t = kmalloc(sizeof *t, GFP_KERNEL); + if (!t) { + ret = -ENOMEM; + goto err_free; + } + + ipoib_cm_init_rx_wr(dev, &t->wr, t->sge); + spin_lock_irq(&priv->lock); if (priv->cm.nonsrq_conn_qp >= ipoib_max_conn_qp) { @@ -349,8 +382,8 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); ret = -ENOMEM; goto err_count; - } - ret = ipoib_cm_post_receive_nonsrq(dev, rx, i); + } + ret = ipoib_cm_post_receive_nonsrq(dev, rx, &t->wr, t->sge, i); if (ret) { ipoib_warn(priv, "ipoib_cm_post_receive_nonsrq " "failed for buf %d\n", i); @@ -361,6 +394,8 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i rx->recv_count = ipoib_recvq_size; + kfree(t); + return 0; err_count: @@ -369,6 +404,7 @@ err_count: spin_unlock_irq(&priv->lock); err_free: + kfree(t); ipoib_cm_free_rx_ring(dev, rx->rx_ring); return ret; @@ -637,7 +673,10 @@ repost: ipoib_warn(priv, "ipoib_cm_post_receive_srq failed " "for buf %d\n", wr_id); } else { - if (unlikely(ipoib_cm_post_receive_nonsrq(dev, p, wr_id))) { + if (unlikely(ipoib_cm_post_receive_nonsrq(dev, p, + &priv->cm.rx_wr, + priv->cm.rx_sge, + wr_id))) { --p->recv_count; ipoib_warn(priv, "ipoib_cm_post_receive_nonsrq failed " "for buf %d\n", wr_id); @@ -1502,15 +1541,7 @@ int ipoib_cm_dev_init(struct net_device *dev) priv->cm.num_frags = IPOIB_CM_RX_SG; } - for (i = 0; i < priv->cm.num_frags; ++i) - priv->cm.rx_sge[i].lkey = priv->mr->lkey; - - priv->cm.rx_sge[0].length = IPOIB_CM_HEAD_SIZE; - for (i = 1; i < priv->cm.num_frags; ++i) - priv->cm.rx_sge[i].length = PAGE_SIZE; - priv->cm.rx_wr.next = NULL; - priv->cm.rx_wr.sg_list = priv->cm.rx_sge; - priv->cm.rx_wr.num_sge = priv->cm.num_frags; + ipoib_cm_init_rx_wr(dev, &priv->cm.rx_wr, priv->cm.rx_sge); if (ipoib_cm_has_srq(dev)) { for (i = 0; i < ipoib_recvq_size; ++i) { -- 1.5.6 From sashak at voltaire.com Wed Jul 2 06:11:04 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 2 Jul 2008 16:11:04 +0300 Subject: [ofa-general] Re: [PATCH RFC] opensm/event_plugin: plugin API version 2 In-Reply-To: <1214861294.3486.216.camel@hrosenstock-ws.xsigo.com> References: <20080625222331.GB28318@sashak.voltaire.com> <1214501218.13056.620.camel@hrosenstock-ws.xsigo.com> <20080627181931.GO29456@sashak.voltaire.com> <20080627130333.08d783c6.weiny2@llnl.gov> <20080627203745.GA29198@obsidianresearch.com> <20080627230339.GV29456@sashak.voltaire.com> <1214825806.3486.173.camel@hrosenstock-ws.xsigo.com> <20080630142705.GF7229@sashak.voltaire.com> <1214861294.3486.216.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080702131104.GI30305@sashak.voltaire.com> Hi Hal, On 14:28 Mon 30 Jun , Hal Rosenstock wrote: > > > > As we learned recently having structured API leads to such problems just > > well, in this case one can expect "stable API" even if it was never > > declared. > > I hardly think that situation is comparable if you are referring to the > library APIs exposed inside of OpenSM for the vendors who wanted these > without OpenSM itself (for diags and ibutils). Why not? We got complains about changes in libosmcomp just few weeks ago. I clearly don't want to make such limitations for OpenSM itself. > > I think I agree with Ira - OpenSM version exact match enforcement will > > make it clearer that plugin writer should be ready to rebuild and > > possibly update its code. > > should being the operative word. > > > > Might this be better handled as packaging with separate packages based > > > on licenses ? > > > > What do you mean? > > This is related to what I wrote below about the OpenFabrics licensing > requirements. The idea is if GPL licensing were to be allowed (perhaps > only in some limited context), then there could be two different > packages: dual and GPL. In that way plugins would be more assured of > being compatible with each other and OpenSM. I would prefer to separate packages by its functionality and not due to licensing issues. Anyway GPL is not permitted in OFA and all this discussion is yet hypothetical. Sasha From ogerlitz at voltaire.com Wed Jul 2 06:24:25 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 02 Jul 2008 16:24:25 +0300 Subject: [ofa-general] Re: [PATCH v4 for-2.6.27] IPOIB: add LRO support. In-Reply-To: References: <4867D58C.5030404@gmail.com> <486A1E29.6080201@voltaire.com> Message-ID: <486B8189.3030507@voltaire.com> Roland Dreier wrote: > Yes, in general I prefer not to have module parameters, especially those > that duplicate functionality available through other mechanisms. > However, having lro and max_aggr parameters seems to match existing > practice for ethernet drivers, and 64 seems to be the common default for max_aggr. OK, I understand that to some extent its fine to do what others (BTW its only 4 drivers under drivers/net that use LRO, so the sample size is kind of small) do. I still don't see why 64 makes sense, for example ipoib supports LSO on the xmit side, which means the network stack would submit SKBs whose data size is up to 64K, assuming the common mtu is 2K-epsilon I would use a value of 64K/2K = ~32 for the max_aggr value, why use 64 ?! Or. From hrosenstock at xsigo.com Wed Jul 2 06:37:15 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 02 Jul 2008 06:37:15 -0700 Subject: [ofa-general] Re: [PATCH RFC] opensm/event_plugin: plugin API version 2 In-Reply-To: <20080702131104.GI30305@sashak.voltaire.com> References: <20080625222331.GB28318@sashak.voltaire.com> <1214501218.13056.620.camel@hrosenstock-ws.xsigo.com> <20080627181931.GO29456@sashak.voltaire.com> <20080627130333.08d783c6.weiny2@llnl.gov> <20080627203745.GA29198@obsidianresearch.com> <20080627230339.GV29456@sashak.voltaire.com> <1214825806.3486.173.camel@hrosenstock-ws.xsigo.com> <20080630142705.GF7229@sashak.voltaire.com> <1214861294.3486.216.camel@hrosenstock-ws.xsigo.com> <20080702131104.GI30305@sashak.voltaire.com> Message-ID: <1215005835.3486.269.camel@hrosenstock-ws.xsigo.com> Hi Sasha, On Wed, 2008-07-02 at 16:11 +0300, Sasha Khapyorsky wrote: > Hi Hal, > > On 14:28 Mon 30 Jun , Hal Rosenstock wrote: > > > > > > As we learned recently having structured API leads to such problems just > > > well, in this case one can expect "stable API" even if it was never > > > declared. > > > > I hardly think that situation is comparable if you are referring to the > > library APIs exposed inside of OpenSM for the vendors who wanted these > > without OpenSM itself (for diags and ibutils). > > Why not? Because exposing all the data structures, etc. is way more granular (all the socalled opaque objects are no longer opaque) than a library API. It's a totally different magnitude IMO. > We got complains about changes in libosmcomp just few weeks > ago. But at least that's way more manageable the way it is and I would have cast the libosmcomp issue in a slightly different light. > I clearly don't want to make such limitations for OpenSM itself. I don't see how those limitations can be made or OpenSM could not move forward. > > > I think I agree with Ira - OpenSM version exact match enforcement will > > > make it clearer that plugin writer should be ready to rebuild and > > > possibly update its code. > > > > should being the operative word. > > > > > > Might this be better handled as packaging with separate packages based > > > > on licenses ? > > > > > > What do you mean? > > > > This is related to what I wrote below about the OpenFabrics licensing > > requirements. The idea is if GPL licensing were to be allowed (perhaps > > only in some limited context), then there could be two different > > packages: dual and GPL. In that way plugins would be more assured of > > being compatible with each other and OpenSM. > > I would prefer to separate packages by its functionality and not due to > licensing issues. Right, licensed based packages do put all related functionality in a similar bucket (management) but is that the basis to make such a decision ? > Anyway GPL is not permitted in OFA and all this > discussion is yet hypothetical. Yes, but one needs to look at hypotheticals to set direction. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Wed Jul 2 06:39:14 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 02 Jul 2008 06:39:14 -0700 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: Only send MCG delete notice if not previously done In-Reply-To: <20080627164812.GJ29456@sashak.voltaire.com> References: <1214516203.13056.654.camel@hrosenstock-ws.xsigo.com> <20080627164812.GJ29456@sashak.voltaire.com> Message-ID: <1215005954.3486.272.camel@hrosenstock-ws.xsigo.com> On Fri, 2008-06-27 at 19:48 +0300, Sasha Khapyorsky wrote: > BTW do you remember what was the reason to not delete MCG on a last > leave request? I know that spec allows this (but requires to post trap > 67 on a last leave), but is there any practical reason for doing this > in OpenSM? >From the spec perspective, there was no need to make it other than lazy as that's the way multicast works so implementations can take advantage of a "convenient" time to do this. I really haven't looked at this specifically in OpenSM and am not sure but think this was added for a practical reason (a real life scenario). I believe Eitan wrote this code so he is probably best to comment and hopefully he will :-) -- Hal From eitan at mellanox.co.il Wed Jul 2 06:44:41 2008 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Wed, 2 Jul 2008 16:44:41 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: Only sendMCG delete notice if not previously done In-Reply-To: <1215005954.3486.272.camel@hrosenstock-ws.xsigo.com> Message-ID: <26E4A768C32B0749989A2C07675EBABD1C3AF1@mtlexch01.mtl.com> In OpenSM we separated "MCG Registration" from "MCG Routing". The reason was that on very large networks it is better to avoid the need to re-compute the tree for each joining node. Instead the MC routing is done for a set of changes. Once a MCG group is deleted by the last member leaving the group (immediately) it is required to still exist until the routing engine deletes its routing. We mark the group with "to be deleted" flag so we know it will be deleted soon. EZ > -----Original Message----- > From: Hal Rosenstock [mailto:hrosenstock at xsigo.com] > Sent: ד 02 יולי 2008 16:39 > To: Sasha Khapyorsky > Cc: general at lists.openfabrics.org; Eitan Zahavi > Subject: Re: [ofa-general] Re: [PATCH] > opensm/osm_mcast_mgr.c: Only sendMCG delete notice if not > previously done > > On Fri, 2008-06-27 at 19:48 +0300, Sasha Khapyorsky wrote: > > BTW do you remember what was the reason to not delete MCG on a last > > leave request? I know that spec allows this (but requires > to post trap > > 67 on a last leave), but is there any practical reason for > doing this > > in OpenSM? > > >From the spec perspective, there was no need to make it > other than lazy > as that's the way multicast works so implementations can take > advantage of a "convenient" time to do this. I really haven't > looked at this specifically in OpenSM and am not sure but > think this was added for a practical reason (a real life scenario). > > I believe Eitan wrote this code so he is probably best to > comment and hopefully he will :-) > > -- Hal > > > From akpm at linux-foundation.org Tue Jul 1 21:44:15 2008 From: akpm at linux-foundation.org (Andrew Morton) Date: Tue, 1 Jul 2008 21:44:15 -0700 Subject: [ofa-general] Re: [PATCH 0 of 3] mmu notifier v18 for -mm In-Reply-To: References: Message-ID: <20080701214415.b3f93706.akpm@linux-foundation.org> On Thu, 26 Jun 2008 02:26:56 +0200 Andrea Arcangeli wrote: > Hello, > > Christoph suggested me to repost v18 for merging in -mm, to give it more > exposure before the .27 merge window opens. There's no code change compared to > the previous v18 submission (the only change is the correction in the comment > in the mm_take_all_locks patch rightfully pointed out by Linus). > > Full patchset including other XPMEM support patches can be found here: > > http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.26-rc7/mmu-notifier-v18 > > Only the three patches of the patchset I'm submitting here by email are ready > for merging, the rest you can find in the website is not ready for merging yet > for various performance degradations, lots of the XPMEM patches needs to be > elaborated to avoid any slowdown for the non-XPMEM case, but I keep > maintaining them to make life easier to XPMEM current development and later we > can keep work on them to make them suitable for inclusion to avoid any > performance degradation risk. I'm a bit concerned about merging the first three patches when there are eleven more patches of which some, afacit, are required to make these three actually useful. Committing these three would be signing a blank cheque. Because if we hit strong objections with the later patches we end up in a cant-go-forward, cant-go-backward situation. So it would be sensible for us all to get down and at least review the whole patch series to satisfy ourselves that this is the direction in which we wish to go. Also, could I ask that you choose better titles for your patches? You'll notice that we never commit patches with titles such as "mm_take_all_locks". Someone (ie: me) will need to chage your patch title to "mmu-notifiers: add mm_take_all_locks() operation" or such. And, sensibly, the patch's filename will be changed to reflect its title - mmu-notifiers-add-mm_take_all_locks-operation.patch And that's OK, that's what they pay me for. But it means that for a period of time, your name for the patch differs from everyone else's name. This gets confusing and can lead to mistakes. Use of consistent naming from end-to-end is better. > (the fourth patch in the series of the above url, is not strictly relealted to > mmu notifiers but it's good at least for me to keep it in the same tree to > test pci-passthrough capable guest running on reserved-ram at the same time of > two regular guests swapping heavily with mmu notifiers which tends to > exercises both spte models at the same time, if you find this confusing I'll > remove it from any later upload, but xpmem users can totally ignore it, it > only touches x86-64 code) From holt at sgi.com Wed Jul 2 01:59:50 2008 From: holt at sgi.com (Robin Holt) Date: Wed, 2 Jul 2008 03:59:50 -0500 Subject: [ofa-general] Re: [PATCH 0 of 3] mmu notifier v18 for -mm In-Reply-To: <486B0B0F.5010605@qumranet.com> References: <20080701214415.b3f93706.akpm@linux-foundation.org> <486B0B0F.5010605@qumranet.com> Message-ID: <20080702085950.GV9696@sgi.com> On Wed, Jul 02, 2008 at 07:58:55AM +0300, Avi Kivity wrote: > Andrew Morton wrote: >> On Thu, 26 Jun 2008 02:26:56 +0200 Andrea Arcangeli wrote: >> >>> Full patchset including other XPMEM support patches can be found here: >>> >>> http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.26-rc7/mmu-notifier-v18 >> >> I'm a bit concerned about merging the first three patches when there >> are eleven more patches of which some, afacit, are required to make >> these three actually useful. Committing these three would be signing a >> blank cheque. >> >> > > The first three are useful for kvm, gru, and likely drm and rdma nics. > > It is only xpmem which requires the other eleven patches. > >> Because if we hit strong objections with the later patches we end up in a >> cant-go-forward, cant-go-backward situation. SGI decided we need a functional GRU more than an enhanced XPMEM. We have, for 7 years now, told our customers that using XPMEM results in pages being permanently pinned and swap being disabled, and they have been accepting of that. We have also put other restrictions on what parts of the address space could be exported. It is not ideal, but it is functional. To work with distro kernels, we have requested 3 EXPORT_SYMBOL_GPL()s from SuSE (SLES-10) and 2 from RedHat (RHEL-5). With that and an LD_PRELOAD library, we can support most types of clone2()s safely and the documentation lists what is not supported (like mmap() into the middle of a region of memory that has been used for calculations). Thanks, Robin Holt From jackm at dev.mellanox.co.il Wed Jul 2 05:10:12 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 2 Jul 2008 15:10:12 +0300 Subject: [ofa-general] ***SPAM*** Re: [PATCH 11/11] mlx4: XRC receive-only QPs In-Reply-To: References: <200806231602.02721.jackm@dev.mellanox.co.il> Message-ID: <200807021510.12761.jackm@dev.mellanox.co.il> On Monday 23 June 2008 19:36, Roland Dreier wrote: > running checkpatch on the kernel patches in this series shows > > total: 30 errors, 89 warnings, 3781 lines checked > > it's a waste of everyone's time to send patches with trivial formatting > problems where there are automated tools to catch them. > Sorry about that. These were old patches (from before I started using checkpatch.pl) -- and I forgot to do this check. I've cleaned them all up -- except for some of the places where I've got lines exceeding 80 chars. - Jack From eli at dev.mellanox.co.il Wed Jul 2 07:03:51 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 2 Jul 2008 17:03:51 +0300 Subject: [ofa-general] Re: [PATCH v4 for-2.6.27] IPOIB: add LRO support. In-Reply-To: <486B8189.3030507@voltaire.com> References: <4867D58C.5030404@gmail.com> <486A1E29.6080201@voltaire.com> <486B8189.3030507@voltaire.com> Message-ID: <20080702140351.GD13098@mtls03> On Wed, Jul 02, 2008 at 04:24:25PM +0300, Or Gerlitz wrote: > OK, I understand that to some extent its fine to do what others (BTW its > only 4 drivers under drivers/net that use LRO, so the sample size is kind > of small) do. I still don't see why 64 makes sense, for example ipoib > supports LSO on the xmit side, which means the network stack would submit > SKBs whose data size is up to 64K, assuming the common mtu is 2K-epsilon I > would use a value of 64K/2K = ~32 for the max_aggr value, why use 64 ?! > I don't see a connection to using or not using LSO -- the 64K comes from the fact that an IP datagram's max length is 64K. And since LRO involves generating at the receiver a hand made IP datagram, the max length is 64K. There is a connection to TCP MSS which could be up to MTU but could be smaller too. I don't see a problem keeping the current value. From jackm at dev.mellanox.co.il Wed Jul 2 08:04:10 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 2 Jul 2008 18:04:10 +0300 Subject: [ofa-general] Re: [PATCH 8/11] core: XRC receive-only QPs In-Reply-To: References: <200806231601.43706.jackm@dev.mellanox.co.il> Message-ID: <200807021804.11083.jackm@dev.mellanox.co.il> On Tuesday 24 June 2008 00:53, Roland Dreier wrote: > > @@ -769,6 +775,7 @@ struct ib_ucontext { > > struct list_head srq_list; > > struct list_head ah_list; > > struct list_head xrc_domain_list; > > + struct list_head xrc_reg_qp_list; > > int closing; > > }; > > Wouldn't it be cleaner to keep the like of recv QPs per xrcd? Then you > wouldn't have to do stuff like: > > > + list_for_each_entry(tmp, &file->ucontext->xrc_reg_qp_list, list) > > + if (cmd.xrcd_handle == tmp->domain_handle) { > > instead you could just do a list_empty() test. > Currently, there is no per-process object for xrc domains. I needed a list of qp's registered ***per-process***, so that I could prevent closing a domain (for that process) while that process still had registered QPs on that domain. This was especially true if the current process was the last one using the domain -- so that the domain itself would be deleted when close_domain was invoked. With the check in place, if the user tried to close the domain while there were QPs registered, he gets an error. If the user does not close the domain, and does not unregister all the QPs for that process, this gets done in ib_uverbs_cleanup_ucontext() (in the proper order), so there are no resource leaks. I think that here we have "6 of one, half a dozen of the other": If I register QPs per xrcd, I either have to create and manage an xrcd user-space specific object (e.g., struct ib_uxrcd_object) -- instead of the generic ib_uobject, or I need to save the process ID per QP in an xrcd list of QPs -- and then I would need to check for equality of PIDs, rather than domain handles. I think the way things are now is relatively simpler. - Jack From 71advertising at cool-telechargement.com Wed Jul 2 06:40:53 2008 From: 71advertising at cool-telechargement.com (emmit jolyon) Date: Wed, 02 Jul 2008 13:40:53 +0000 Subject: [ofa-general] to general Message-ID: <000901c8dc58$04691789$cacf67be@lxauqe> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avawewosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From 87atomlinks at laban-decroux.org Wed Jul 2 06:46:57 2008 From: 87atomlinks at laban-decroux.org (clair freeborn) Date: Wed, 02 Jul 2008 13:46:57 +0000 Subject: [ofa-general] to general Message-ID: <000401c8dc59$0345d3b4$7a744b9a@hkxbro> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avawewosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From shemminger at vyatta.com Wed Jul 2 09:41:40 2008 From: shemminger at vyatta.com (Stephen Hemminger) Date: Wed, 2 Jul 2008 09:41:40 -0700 Subject: [ofa-general] Re: locking requirements when calling dev_set_mtu() In-Reply-To: <20080702085409.GA13098@mtls03> References: <20080702085409.GA13098@mtls03> Message-ID: <20080702094140.032efd28@extreme> On Wed, 2 Jul 2008 11:54:09 +0300 Eli Cohen wrote: > Hi, > > I have the need to change the MTU of IP over IB from the driver code > and not from the shell. Looking for the right way to do it I came > across the dev_set_mtu() which seems appropriate as it handles all > notifications too. However, it is not clear to me whether I have to > wrap the call to dev_set_mtu() with any locks. Specifically, I was > referred to a mail thread that suggests that I have to use > rtnl_lock()/rtnl_unlock(): > http://marc.info/?l=linux-netdev&m=121201324611292&w=2 > > With reference to the above URL, I would say that it is possible for > me to hit to the assertion ASSERT_RTNL() if I don't take rtnl_lock > prior to calling dev_set_mtu(). However I don't hit it, though I can't > count on that there will not be a case that any function registered > for notification would expect to see the lock acquired. > You need to call rtnl_lock() so device drivers get consistent locking. From jeff at splitrockpr.com Wed Jul 2 09:46:39 2008 From: jeff at splitrockpr.com (Jeffrey Scott) Date: Wed, 02 Jul 2008 09:46:39 -0700 Subject: [ofa-general] Save the Date: IBTA Technical Forum 2008 Message-ID: <352E1EEDDABC4D52992EFD44A0106FD2@Gaucho> Attention OFA Members: IBTA Technical Forum 2008 Please save the date for the IBTA's annual technical forum, which is being held this year on September 15, 2008 at Harrah's in Las Vegas! The "IBTA Technical Forum '08: InfiniBand and the Enterprise Data Center" will take place from 8 a.m. - 5 p.m. and will feature John Enck, research vice president at Gartner, Inc, as the keynote speaker. A networking reception will follow at the end of the day. To register for this event, please visit http://www.infinibandta.org/events/IBTATechForum08_. Sponsorship Opportunity For $1,000, your company can co-sponsor the technical forum's networking reception. This sponsorship opportunity is a great way to gain exposure to high-level industry experts and decision makers from the InfiniBand industry. Each sponsor will receive recognition on reception signage and the opportunity to display company marketing materials in the reception area. For more information on sponsorships, please contact: Samantha Spears, Owen Media, SamanthaS at owenmedia.com. Call for Speakers: The IBTA is also announcing speaker opportunities for this year's event. If you know an end user / InfiniBand customer with a virtualization or Oracle-based deployment who may be appropriate and available to speak at this year's technical forum, please send a proposed abstract and contact information to: Samantha Spears, Owen Media, SamanthaS at owenmedia.com. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bhutchings at solarflare.com Wed Jul 2 10:15:52 2008 From: bhutchings at solarflare.com (Ben Hutchings) Date: Wed, 2 Jul 2008 18:15:52 +0100 Subject: [ofa-general] Re: locking requirements when calling dev_set_mtu() In-Reply-To: <20080702085409.GA13098@mtls03> References: <20080702085409.GA13098@mtls03> Message-ID: <20080702171550.GE17872@solarflare.com> Eli Cohen wrote: > Hi, > > I have the need to change the MTU of IP over IB from the driver code > and not from the shell. Looking for the right way to do it I came > across the dev_set_mtu() which seems appropriate as it handles all > notifications too. However, it is not clear to me whether I have to > wrap the call to dev_set_mtu() with any locks. Specifically, I was > referred to a mail thread that suggests that I have to use > rtnl_lock()/rtnl_unlock(): > http://marc.info/?l=linux-netdev&m=121201324611292&w=2 [...] Essentially any function that changes net device configuration requires the caller to hold the RTNL lock. This probably ought to be consistently documented through kernel-doc and/or ASSERT_RTNL() though. Ben. -- Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. From sean.hefty at intel.com Wed Jul 2 11:48:07 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Wed, 2 Jul 2008 11:48:07 -0700 Subject: [ofa-general] cma: add support for TIMEWAIT_EXIT event (second try) In-Reply-To: <48590FF2.8090507@mellanox.co.il> References: <48590FF2.8090507@mellanox.co.il> Message-ID: <001201c8dc74$2b8c6f40$b2fc070a@amr.corp.intel.com> >SDP needs to be notified when CM exits the TIMEWAIT state. >CM does notify the CMA but CMA doesn't pass it to the upper layer. > >Below is a patch for the CMA code. I wasn't sure if it is ok to set >CMA_DISCONNECT in the IB_CM_TIMEWAIT_EXIT instead of doing it in the >IB_CM_DREQ_RECEIVED/IB_CM_DREP_RECEIVED as it is done today. Instead >I changed the check at the beginning of cma_ib_handler() as you can >see in the patch. > >Signed-off-by: Amir Vadai Sorry I was slow to get to this. The general approach looks fine to me. >--- include/rdma/rdma_cm.h.orig 2008-06-18 15:04:22.560904000 +0300 >+++ include/rdma/rdma_cm.h 2008-06-11 11:55:26.758053000 +0300 >@@ -53,7 +53,8 @@ enum rdma_cm_event_type { > RDMA_CM_EVENT_DISCONNECTED, > RDMA_CM_EVENT_DEVICE_REMOVAL, > RDMA_CM_EVENT_MULTICAST_JOIN, >- RDMA_CM_EVENT_MULTICAST_ERROR >+ RDMA_CM_EVENT_MULTICAST_ERROR, >+ RDMA_CM_EVENT_TIMWAIT_EXIT > }; This will need to be updated based on the address change event patch that went in. I'll update the librdmacm myself separately. > enum rdma_port_space { >--- drivers/infiniband/core/cma.c.orig 2008-06-11 11:24:38.021543000 >+0300 >+++ drivers/infiniband/core/cma.c 2008-06-18 13:27:08.098747000 +0300 >@@ -931,7 +931,8 @@ static int cma_ib_handler(struct ib_cm_i > struct rdma_cm_event event; > int ret = 0; > >- if (cma_disable_remove(id_priv, CMA_CONNECT)) >+ if ((ib_event->event != IB_CM_TIMEWAIT_EXIT && >cma_disable_remove(id_priv, CMA_CONNECT)) || >+ (ib_event->event == IB_CM_TIMEWAIT_EXIT && >cma_disable_remove(id_priv, CMA_DISCONNECT))) > return 0; I wonder if it would be better to acquire the handler_mutex explicitly, like this: mutex_lock(&id_priv->handler_mutex); if ((id_priv->state == CMA_CONNECT && ib_event->event != IB_CM_TIMEWAIT_EXIT) || (id_priv->state == CMA_DISCONNECT && ib_event->event == IB_CM_TIMEWAIT_EXIT)) { mutex_unlock(&id_priv->handler_mutex); return 0; } so that ensuring the mutex is acquired once isn't so subtle. - Sean From harms at mcs.anl.gov Wed Jul 2 13:13:20 2008 From: harms at mcs.anl.gov (Kevin Harms) Date: Wed, 2 Jul 2008 15:13:20 -0500 (CDT) Subject: [ofa-general] SRP and sync In-Reply-To: <11704585.916951215029522626.JavaMail.root@zimbra> Message-ID: <31670049.917051215029600567.JavaMail.root@zimbra> I'm doing some testing of direct attached storage using SRP. When I use 'sync', the write performance drops off significantly below what the device is capable of for a given sustained write. The read shows something more along the lines of the expected performance. I'm using OFED 1.2.5.5. Is there some issue with OFED and using sync? thanks, Kevin Harms fs121: # sg_dd if=/dev/zero of=dd.test oflag=dsync bs=4096 bpt=512 count=10000 time=1 time to transfer data: 0.721404 secs at 56.78 MB/sec 10000+0 records in 10000+0 records out fs121: # sync fs121: # sysctl -w vm.drop_caches=1 vm.drop_caches = 1 fs121: # sysctl -w vm.drop_caches=2 vm.drop_caches = 2 fs121: # sysctl -w vm.drop_caches=3 vm.drop_caches = 3 fs121: # sg_dd of=/dev/zero if=dd.test bs=4096 bpt=512 count=10000 time=1 time to transfer data: 0.091308 secs at 448.59 MB/sec 10000+0 records in 10000+0 records out From rdreier at cisco.com Wed Jul 2 14:01:01 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 02 Jul 2008 14:01:01 -0700 Subject: [ofa-general] [PATCH] RDMA/nes: Get rid of ring_doorbell parameter of nes_post_cqp_request() Message-ID: Every caller of nes_post_cqp_request() passed it NES_CQP_REQUEST_RING_DOORBELL, so just remove that parameter and always ring the doorbell. Signed-off-by: Roland Dreier --- And one more... drivers/infiniband/hw/nes/nes.c | 2 +- drivers/infiniband/hw/nes/nes.h | 5 +---- drivers/infiniband/hw/nes/nes_hw.c | 6 +++--- drivers/infiniband/hw/nes/nes_utils.c | 9 ++++----- drivers/infiniband/hw/nes/nes_verbs.c | 18 +++++++++--------- 5 files changed, 18 insertions(+), 22 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes.c b/drivers/infiniband/hw/nes/nes.c index a4e9269..d2884e7 100644 --- a/drivers/infiniband/hw/nes/nes.c +++ b/drivers/infiniband/hw/nes/nes.c @@ -328,7 +328,7 @@ void nes_rem_ref(struct ib_qp *ibqp) set_wqe_32bit_value(cqp_wqe->wqe_words, NES_CQP_WQE_ID_IDX, nesqp->hwqp.qp_id); u64temp = (u64)nesqp->nesqp_context_pbase; set_wqe_64bit_value(cqp_wqe->wqe_words, NES_CQP_QP_WQE_CONTEXT_LOW_IDX, u64temp); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); } } diff --git a/drivers/infiniband/hw/nes/nes.h b/drivers/infiniband/hw/nes/nes.h index fe88bec..39bd897 100644 --- a/drivers/infiniband/hw/nes/nes.h +++ b/drivers/infiniband/hw/nes/nes.h @@ -94,9 +94,6 @@ #define MAX_DPC_ITERATIONS 128 -#define NES_CQP_REQUEST_NO_DOORBELL_RING 0 -#define NES_CQP_REQUEST_RING_DOORBELL 1 - #define NES_DRV_OPT_ENABLE_MPA_VER_0 0x00000001 #define NES_DRV_OPT_DISABLE_MPA_CRC 0x00000002 #define NES_DRV_OPT_DISABLE_FIRST_WRITE 0x00000004 @@ -542,7 +539,7 @@ void nes_free_cqp_request(struct nes_device *nesdev, struct nes_cqp_request *cqp_request); void nes_put_cqp_request(struct nes_device *nesdev, struct nes_cqp_request *cqp_request); -void nes_post_cqp_request(struct nes_device *, struct nes_cqp_request *, int); +void nes_post_cqp_request(struct nes_device *, struct nes_cqp_request *); int nes_arp_table(struct nes_device *, u32, u8 *, u32); void nes_mh_fix(unsigned long); void nes_clc(unsigned long); diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c index 0cea58b..59c2841 100644 --- a/drivers/infiniband/hw/nes/nes_hw.c +++ b/drivers/infiniband/hw/nes/nes_hw.c @@ -3145,7 +3145,7 @@ int nes_manage_apbvt(struct nes_vnic *nesvnic, u32 accel_local_port, nes_debug(NES_DBG_QP, "Waiting for CQP completion for APBVT.\n"); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); if (add_port == NES_MANAGE_APBVT_ADD) ret = wait_event_timeout(cqp_request->waitq, (cqp_request->request_done != 0), @@ -3215,7 +3215,7 @@ void nes_manage_arp_cache(struct net_device *netdev, unsigned char *mac_addr, nesdev->cqp.sq_head, nesdev->cqp.sq_tail); atomic_set(&cqp_request->refcount, 1); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); } @@ -3247,7 +3247,7 @@ void flush_wqes(struct nes_device *nesdev, struct nes_qp *nesqp, cpu_to_le32(NES_CQP_FLUSH_WQES | which_wq); cqp_wqe->wqe_words[NES_CQP_WQE_ID_IDX] = cpu_to_le32(nesqp->hwqp.qp_id); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); if (wait_completion) { /* Wait for CQP */ diff --git a/drivers/infiniband/hw/nes/nes_utils.c b/drivers/infiniband/hw/nes/nes_utils.c index 8f519c8..fb8cbd7 100644 --- a/drivers/infiniband/hw/nes/nes_utils.c +++ b/drivers/infiniband/hw/nes/nes_utils.c @@ -596,7 +596,7 @@ void nes_put_cqp_request(struct nes_device *nesdev, * nes_post_cqp_request */ void nes_post_cqp_request(struct nes_device *nesdev, - struct nes_cqp_request *cqp_request, int ring_doorbell) + struct nes_cqp_request *cqp_request) { struct nes_hw_cqp_wqe *cqp_wqe; unsigned long flags; @@ -624,10 +624,9 @@ void nes_post_cqp_request(struct nes_device *nesdev, nesdev->cqp.sq_head, nesdev->cqp.sq_tail, nesdev->cqp.sq_size, cqp_request->waiting, atomic_read(&cqp_request->refcount)); barrier(); - if (ring_doorbell) { - /* Ring doorbell (1 WQEs) */ - nes_write32(nesdev->regs+NES_WQE_ALLOC, 0x01800000 | nesdev->cqp.qp_id); - } + + /* Ring doorbell (1 WQEs) */ + nes_write32(nesdev->regs+NES_WQE_ALLOC, 0x01800000 | nesdev->cqp.qp_id); barrier(); } else { diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index 464a98a..e3939d1 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -118,7 +118,7 @@ static struct ib_mw *nes_alloc_mw(struct ib_pd *ibpd) { set_wqe_32bit_value(cqp_wqe->wqe_words, NES_CQP_STAG_WQE_STAG_IDX, stag); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ ret = wait_event_timeout(cqp_request->waitq, (cqp_request->request_done != 0), @@ -175,7 +175,7 @@ static int nes_dealloc_mw(struct ib_mw *ibmw) set_wqe_32bit_value(cqp_wqe->wqe_words, NES_CQP_STAG_WQE_STAG_IDX, ibmw->rkey); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ nes_debug(NES_DBG_MR, "Waiting for deallocate STag 0x%08X to complete.\n", @@ -477,7 +477,7 @@ static struct ib_fmr *nes_alloc_fmr(struct ib_pd *ibpd, (nesfmr->nesmr.pbls_used-1) : nesfmr->nesmr.pbls_used); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ ret = wait_event_timeout(cqp_request->waitq, (cqp_request->request_done != 0), @@ -1417,7 +1417,7 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd, set_wqe_64bit_value(cqp_wqe->wqe_words, NES_CQP_QP_WQE_CONTEXT_LOW_IDX, u64temp); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ nes_debug(NES_DBG_QP, "Waiting for create iWARP QP%u to complete.\n", @@ -1744,7 +1744,7 @@ static struct ib_cq *nes_create_cq(struct ib_device *ibdev, int entries, cpu_to_le32(((u32)((u64temp) >> 33)) & 0x7FFFFFFF); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ nes_debug(NES_DBG_CQ, "Waiting for create iWARP CQ%u to complete.\n", @@ -1841,7 +1841,7 @@ static int nes_destroy_cq(struct ib_cq *ib_cq) (nescq->hw_cq.cq_number | ((u32)PCI_FUNC(nesdev->pcidev->devfn) << 16))); nes_free_resource(nesadapter, nesadapter->allocated_cqs, nescq->hw_cq.cq_number); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ nes_debug(NES_DBG_CQ, "Waiting for destroy iWARP CQ%u to complete.\n", @@ -1987,7 +1987,7 @@ static int nes_reg_mr(struct nes_device *nesdev, struct nes_pd *nespd, barrier(); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ ret = wait_event_timeout(cqp_request->waitq, (0 != cqp_request->request_done), @@ -2638,7 +2638,7 @@ static int nes_dereg_mr(struct ib_mr *ib_mr) set_wqe_32bit_value(cqp_wqe->wqe_words, NES_CQP_STAG_WQE_STAG_IDX, ib_mr->rkey); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ nes_debug(NES_DBG_MR, "Waiting for deallocate STag 0x%08X completed\n", ib_mr->rkey); @@ -2809,7 +2809,7 @@ int nes_hw_modify_qp(struct nes_device *nesdev, struct nes_qp *nesqp, set_wqe_64bit_value(cqp_wqe->wqe_words, NES_CQP_QP_WQE_CONTEXT_LOW_IDX, (u64)nesqp->nesqp_context_pbase); atomic_set(&cqp_request->refcount, 2); - nes_post_cqp_request(nesdev, cqp_request, NES_CQP_REQUEST_RING_DOORBELL); + nes_post_cqp_request(nesdev, cqp_request); /* Wait for CQP */ if (wait_completion) { -- 1.5.6 From flatif at NetEffect.com Wed Jul 2 15:17:06 2008 From: flatif at NetEffect.com (Faisal Latif) Date: Wed, 2 Jul 2008 17:17:06 -0500 Subject: [ofa-general] RE: [PATCH] RDMA/nes: Encapsulate logic nes_put_cqp_request() In-Reply-To: References: Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC0829CC02@venom2> Acked-by: Faisal Latif Thanks Faisal > -----Original Message----- > From: Roland Dreier [mailto:rdreier at cisco.com] > Sent: Tuesday, July 01, 2008 11:07 PM > To: Faisal Latif; Chien Tung; Glenn Streiff > Cc: general at lists.openfabrics.org > Subject: [PATCH] RDMA/nes: Encapsulate logic nes_put_cqp_request() > > The iw_nes driver repeats the logic > > if (atomic_dec_and_test(&cqp_request->refcount)) { > if (cqp_request->dynamic) { > kfree(cqp_request); > } else { > spin_lock_irqsave(&nesdev->cqp.lock, flags); > list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > spin_unlock_irqrestore(&nesdev->cqp.lock, flags); > } > } > > over and over. Wrap this up in functions nes_free_cqp_request() and > nes_put_cqp_request() to simplify such code. > > In addition to making the source smaller and more readable, this > shrinks > the compiled code quite a bit: > > add/remove: 2/0 grow/shrink: 0/13 up/down: 164/-1692 (-1528) > function old new delta > nes_free_cqp_request - 147 +147 > nes_put_cqp_request - 17 +17 > nes_modify_qp 2316 2293 -23 > nes_hw_modify_qp 737 657 -80 > nes_dereg_mr 945 860 -85 > flush_wqes 501 416 -85 > nes_manage_apbvt 648 560 -88 > nes_reg_mr 1117 1026 -91 > nes_cqp_ce_handler 927 769 -158 > nes_alloc_mw 1052 884 -168 > nes_create_qp 5314 5141 -173 > nes_alloc_fmr 2212 2035 -177 > nes_destroy_cq 1097 918 -179 > nes_create_cq 2787 2598 -189 > nes_dealloc_mw 762 566 -196 > > Signed-off-by: Roland Dreier > --- > Here's a cleanup I'll queue for 2.6.27 if it's OK with you. > > drivers/infiniband/hw/nes/nes.h | 4 + > drivers/infiniband/hw/nes/nes_hw.c | 60 ++--------- > drivers/infiniband/hw/nes/nes_utils.c | 24 ++++ > drivers/infiniband/hw/nes/nes_verbs.c | 189 +++++-------------------- > ------- > 4 files changed, 66 insertions(+), 211 deletions(-) > > diff --git a/drivers/infiniband/hw/nes/nes.h > b/drivers/infiniband/hw/nes/nes.h > index 61b46e9..fe88bec 100644 > --- a/drivers/infiniband/hw/nes/nes.h > +++ b/drivers/infiniband/hw/nes/nes.h > @@ -538,6 +538,10 @@ void nes_read_1G_phy_reg(struct nes_device *, u8, > u8, u16 *); > void nes_write_10G_phy_reg(struct nes_device *, u16, u8, u16, u16); > void nes_read_10G_phy_reg(struct nes_device *, u8, u8, u16); > struct nes_cqp_request *nes_get_cqp_request(struct nes_device *); > +void nes_free_cqp_request(struct nes_device *nesdev, > + struct nes_cqp_request *cqp_request); > +void nes_put_cqp_request(struct nes_device *nesdev, > + struct nes_cqp_request *cqp_request); > void nes_post_cqp_request(struct nes_device *, struct nes_cqp_request > *, int); > int nes_arp_table(struct nes_device *, u32, u8 *, u32); > void nes_mh_fix(unsigned long); > diff --git a/drivers/infiniband/hw/nes/nes_hw.c > b/drivers/infiniband/hw/nes/nes_hw.c > index d3278f1..80e4866 100644 > --- a/drivers/infiniband/hw/nes/nes_hw.c > +++ b/drivers/infiniband/hw/nes/nes_hw.c > @@ -2710,39 +2710,11 @@ static void nes_cqp_ce_handler(struct > nes_device *nesdev, struct nes_hw_cq *cq) > barrier(); > cqp_request->request_done = 1; > wake_up(&cqp_request->waitq); > - if (atomic_dec_and_test(&cqp_request- > >refcount)) { > - nes_debug(NES_DBG_CQP, "CQP request > %p (opcode 0x%02X) freed.\n", > - cqp_request, > - > le32_to_cpu(cqp_request- > >cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX])&0x3f); > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev- > >cqp.lock, flags); > - list_add_tail(&cqp_request- > >list, &nesdev->cqp_avail_reqs); > - > spin_unlock_irqrestore(&nesdev->cqp.lock, flags); > - } > - } > - } else if (cqp_request->callback) { > - /* Envoke the callback routine */ > - cqp_request->cqp_callback(nesdev, > cqp_request); > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev- > >cqp.lock, flags); > - list_add_tail(&cqp_request->list, > &nesdev->cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev- > >cqp.lock, flags); > - } > + nes_put_cqp_request(nesdev, cqp_request); > } else { > - nes_debug(NES_DBG_CQP, "CQP request %p > (opcode 0x%02X) freed.\n", > - cqp_request, > - le32_to_cpu(cqp_request- > >cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX]) & 0x3f); > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev- > >cqp.lock, flags); > - list_add_tail(&cqp_request->list, > &nesdev->cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev- > >cqp.lock, flags); > - } > + if (cqp_request->callback) > + cqp_request->cqp_callback(nesdev, > cqp_request); > + nes_free_cqp_request(nesdev, > cqp_request); > } > } else { > wake_up(&nesdev->cqp.waitq); > @@ -3149,7 +3121,6 @@ int nes_manage_apbvt(struct nes_vnic *nesvnic, > u32 accel_local_port, > { > struct nes_device *nesdev = nesvnic->nesdev; > struct nes_hw_cqp_wqe *cqp_wqe; > - unsigned long flags; > struct nes_cqp_request *cqp_request; > int ret = 0; > u16 major_code; > @@ -3184,15 +3155,9 @@ int nes_manage_apbvt(struct nes_vnic *nesvnic, > u32 accel_local_port, > nes_debug(NES_DBG_QP, "Completed, ret=%u, CQP Major:Minor codes > = 0x%04X:0x%04X\n", > ret, cqp_request->major_code, cqp_request- > >minor_code); > major_code = cqp_request->major_code; > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); > - } > - } > + > + nes_put_cqp_request(nesdev, cqp_request); > + > if (!ret) > return -ETIME; > else if (major_code) > @@ -3262,7 +3227,6 @@ void nes_manage_arp_cache(struct net_device > *netdev, unsigned char *mac_addr, > void flush_wqes(struct nes_device *nesdev, struct nes_qp *nesqp, > u32 which_wq, u32 wait_completion) > { > - unsigned long flags; > struct nes_cqp_request *cqp_request; > struct nes_hw_cqp_wqe *cqp_wqe; > int ret; > @@ -3294,14 +3258,6 @@ void flush_wqes(struct nes_device *nesdev, > struct nes_qp *nesqp, > nes_debug(NES_DBG_QP, "Flush SQ QP WQEs completed, ret=%u," > " CQP Major:Minor codes = 0x%04X:0x%04X\n", > ret, cqp_request->major_code, cqp_request- > >minor_code); > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > + nes_put_cqp_request(nesdev, cqp_request); > } > } > diff --git a/drivers/infiniband/hw/nes/nes_utils.c > b/drivers/infiniband/hw/nes/nes_utils.c > index fe83d1b..82abe9b 100644 > --- a/drivers/infiniband/hw/nes/nes_utils.c > +++ b/drivers/infiniband/hw/nes/nes_utils.c > @@ -567,6 +567,30 @@ struct nes_cqp_request *nes_get_cqp_request(struct > nes_device *nesdev) > return cqp_request; > } > > +void nes_free_cqp_request(struct nes_device *nesdev, > + struct nes_cqp_request *cqp_request) > +{ > + unsigned long flags; > + > + nes_debug(NES_DBG_CQP, "CQP request %p (opcode 0x%02X) freed.\n", > + cqp_request, > + le32_to_cpu(cqp_request- > >cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX])&0x3f); > + > + if (cqp_request->dynamic) { > + kfree(cqp_request); > + } else { > + spin_lock_irqsave(&nesdev->cqp.lock, flags); > + list_add_tail(&cqp_request->list, &nesdev->cqp_avail_reqs); > + spin_unlock_irqrestore(&nesdev->cqp.lock, flags); > + } > +} > + > +void nes_put_cqp_request(struct nes_device *nesdev, > + struct nes_cqp_request *cqp_request) > +{ > + if (atomic_dec_and_test(&cqp_request->refcount)) > + nes_free_cqp_request(nesdev, cqp_request); > +} > > /** > * nes_post_cqp_request > diff --git a/drivers/infiniband/hw/nes/nes_verbs.c > b/drivers/infiniband/hw/nes/nes_verbs.c > index d617da9..464a98a 100644 > --- a/drivers/infiniband/hw/nes/nes_verbs.c > +++ b/drivers/infiniband/hw/nes/nes_verbs.c > @@ -55,7 +55,6 @@ static void nes_unregister_ofa_device(struct > nes_ib_device *nesibdev); > * nes_alloc_mw > */ > static struct ib_mw *nes_alloc_mw(struct ib_pd *ibpd) { > - unsigned long flags; > struct nes_pd *nespd = to_nespd(ibpd); > struct nes_vnic *nesvnic = to_nesvnic(ibpd->device); > struct nes_device *nesdev = nesvnic->nesdev; > @@ -128,15 +127,7 @@ static struct ib_mw *nes_alloc_mw(struct ib_pd > *ibpd) { > " CQP Major:Minor codes = 0x%04X:0x%04X.\n", > stag, ret, cqp_request->major_code, cqp_request- > >minor_code); > if ((!ret) || (cqp_request->major_code)) { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > + nes_put_cqp_request(nesdev, cqp_request); > kfree(nesmr); > nes_free_resource(nesadapter, nesadapter->allocated_mrs, > stag_index); > if (!ret) { > @@ -144,17 +135,8 @@ static struct ib_mw *nes_alloc_mw(struct ib_pd > *ibpd) { > } else { > return ERR_PTR(-ENOMEM); > } > - } else { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > } > + nes_put_cqp_request(nesdev, cqp_request); > > nesmr->ibmw.rkey = stag; > nesmr->mode = IWNES_MEMREG_TYPE_MW; > @@ -178,7 +160,6 @@ static int nes_dealloc_mw(struct ib_mw *ibmw) > struct nes_hw_cqp_wqe *cqp_wqe; > struct nes_cqp_request *cqp_request; > int err = 0; > - unsigned long flags; > int ret; > > /* Deallocate the window with the adapter */ > @@ -204,32 +185,12 @@ static int nes_dealloc_mw(struct ib_mw *ibmw) > nes_debug(NES_DBG_MR, "Deallocate STag completed, > wait_event_timeout ret = %u," > " CQP Major:Minor codes = 0x%04X:0x%04X.\n", > ret, cqp_request->major_code, cqp_request- > >minor_code); > - if ((!ret) || (cqp_request->major_code)) { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > - if (!ret) { > - err = -ETIME; > - } else { > - err = -EIO; > - } > - } else { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > - } > + if (!ret) > + err = -ETIME; > + else if (cqp_request->major_code) > + err = -EIO; > + > + nes_put_cqp_request(nesdev, cqp_request); > > nes_free_resource(nesadapter, nesadapter->allocated_mrs, > (ibmw->rkey & 0x0fffff00) >> 8); > @@ -526,29 +487,11 @@ static struct ib_fmr *nes_alloc_fmr(struct ib_pd > *ibpd, > stag, ret, cqp_request->major_code, cqp_request- > >minor_code); > > if ((!ret) || (cqp_request->major_code)) { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > + nes_put_cqp_request(nesdev, cqp_request); > ret = (!ret) ? -ETIME : -EIO; > goto failed_leaf_vpbl_pages_alloc; > - } else { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > } > - > + nes_put_cqp_request(nesdev, cqp_request); > nesfmr->nesmr.ibfmr.lkey = stag; > nesfmr->nesmr.ibfmr.rkey = stag; > nesfmr->attr = *ibfmr_attr; > @@ -1487,15 +1430,7 @@ static struct ib_qp *nes_create_qp(struct ib_pd > *ibpd, > nesqp->hwqp.qp_id, ret, nesdev- > >cqp.sq_head, nesdev->cqp.sq_tail, > cqp_request->major_code, cqp_request- > >minor_code); > if ((!ret) || (cqp_request->major_code)) { > - if (atomic_dec_and_test(&cqp_request- > >refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev- > >cqp.lock, flags); > - list_add_tail(&cqp_request->list, > &nesdev->cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev- > >cqp.lock, flags); > - } > - } > + nes_put_cqp_request(nesdev, cqp_request); > nes_free_resource(nesadapter, nesadapter- > >allocated_qps, qp_num); > nes_free_qp_mem(nesdev, nesqp,virt_wqs); > kfree(nesqp->allocated_buffer); > @@ -1504,18 +1439,10 @@ static struct ib_qp *nes_create_qp(struct ib_pd > *ibpd, > } else { > return ERR_PTR(-EIO); > } > - } else { > - if (atomic_dec_and_test(&cqp_request- > >refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev- > >cqp.lock, flags); > - list_add_tail(&cqp_request->list, > &nesdev->cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev- > >cqp.lock, flags); > - } > - } > } > > + nes_put_cqp_request(nesdev, cqp_request); > + > if (ibpd->uobject) { > uresp.mmap_sq_db_index = nesqp- > >mmap_sq_db_index; > uresp.actual_sq_size = sq_size; > @@ -1827,32 +1754,15 @@ static struct ib_cq *nes_create_cq(struct > ib_device *ibdev, int entries, > nes_debug(NES_DBG_CQ, "Create iWARP CQ%u completed, > wait_event_timeout ret = %d.\n", > nescq->hw_cq.cq_number, ret); > if ((!ret) || (cqp_request->major_code)) { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > + nes_put_cqp_request(nesdev, cqp_request); > if (!context) > pci_free_consistent(nesdev->pcidev, nescq- > >cq_mem_size, mem, > nescq->hw_cq.cq_pbase); > nes_free_resource(nesadapter, nesadapter->allocated_cqs, > cq_num); > kfree(nescq); > return ERR_PTR(-EIO); > - } else { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > } > + nes_put_cqp_request(nesdev, cqp_request); > > if (context) { > /* free the nespbl */ > @@ -1942,37 +1852,18 @@ static int nes_destroy_cq(struct ib_cq *ib_cq) > " CQP Major:Minor codes = 0x%04X:0x%04X.\n", > nescq->hw_cq.cq_number, ret, cqp_request->major_code, > cqp_request->minor_code); > - if ((!ret) || (cqp_request->major_code)) { > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > - if (!ret) { > - nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy timeout > expired\n", > + if (!ret) { > + nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy timeout > expired\n", > nescq->hw_cq.cq_number); > - ret = -ETIME; > - } else { > - nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy failed\n", > + ret = -ETIME; > + } else if (cqp_request->major_code) { > + nes_debug(NES_DBG_CQ, "iWARP CQ%u destroy failed\n", > nescq->hw_cq.cq_number); > - ret = -EIO; > - } > + ret = -EIO; > } else { > ret = 0; > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > } > + nes_put_cqp_request(nesdev, cqp_request); > > if (nescq->cq_mem_size) > pci_free_consistent(nesdev->pcidev, nescq->cq_mem_size, > @@ -2105,15 +1996,8 @@ static int nes_reg_mr(struct nes_device *nesdev, > struct nes_pd *nespd, > " CQP Major:Minor codes = 0x%04X:0x%04X.\n", > stag, ret, cqp_request->major_code, cqp_request- > >minor_code); > major_code = cqp_request->major_code; > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); > - } > - } > + nes_put_cqp_request(nesdev, cqp_request); > + > if (!ret) > return -ETIME; > else if (major_code) > @@ -2771,15 +2655,9 @@ static int nes_dereg_mr(struct ib_mr *ib_mr) > > major_code = cqp_request->major_code; > minor_code = cqp_request->minor_code; > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, flags); > - } > - } > + > + nes_put_cqp_request(nesdev, cqp_request); > + > if (!ret) { > nes_debug(NES_DBG_MR, "Timeout waiting to destroy STag," > " ib_mr=%p, rkey = 0x%08X\n", > @@ -2904,7 +2782,6 @@ int nes_hw_modify_qp(struct nes_device *nesdev, > struct nes_qp *nesqp, > /* struct iw_cm_id *cm_id = nesqp->cm_id; */ > /* struct iw_cm_event cm_event; */ > struct nes_cqp_request *cqp_request; > - unsigned long flags; > int ret; > u16 major_code; > > @@ -2950,15 +2827,9 @@ int nes_hw_modify_qp(struct nes_device *nesdev, > struct nes_qp *nesqp, > nesqp->hwqp.qp_id, cqp_request- > >major_code, > cqp_request->minor_code, > next_iwarp_state); > } > - if (atomic_dec_and_test(&cqp_request->refcount)) { > - if (cqp_request->dynamic) { > - kfree(cqp_request); > - } else { > - spin_lock_irqsave(&nesdev->cqp.lock, flags); > - list_add_tail(&cqp_request->list, &nesdev- > >cqp_avail_reqs); > - spin_unlock_irqrestore(&nesdev->cqp.lock, > flags); > - } > - } > + > + nes_put_cqp_request(nesdev, cqp_request); > + > if (!ret) > return -ETIME; > else if (major_code) > -- > 1.5.6 From flatif at NetEffect.com Wed Jul 2 15:21:35 2008 From: flatif at NetEffect.com (Faisal Latif) Date: Wed, 2 Jul 2008 17:21:35 -0500 Subject: [ofa-general] RE: [PATCH] RDMA/nes: Get rid of ring_doorbell parameter of nes_post_cqp_request() In-Reply-To: References: Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC0829CC06@venom2> Acked-by: Faisal Latif Thanks Faisal > -----Original Message----- > From: Roland Dreier [mailto:rdreier at cisco.com] > Sent: Wednesday, July 02, 2008 4:01 PM > To: Faisal Latif; Chien Tung; Glenn Streiff > Cc: general at lists.openfabrics.org > Subject: [PATCH] RDMA/nes: Get rid of ring_doorbell parameter of > nes_post_cqp_request() > > Every caller of nes_post_cqp_request() passed it > NES_CQP_REQUEST_RING_DOORBELL, > so just remove that parameter and always ring the doorbell. > > Signed-off-by: Roland Dreier > --- > And one more... > > drivers/infiniband/hw/nes/nes.c | 2 +- > drivers/infiniband/hw/nes/nes.h | 5 +---- > drivers/infiniband/hw/nes/nes_hw.c | 6 +++--- > drivers/infiniband/hw/nes/nes_utils.c | 9 ++++----- > drivers/infiniband/hw/nes/nes_verbs.c | 18 +++++++++--------- > 5 files changed, 18 insertions(+), 22 deletions(-) > > diff --git a/drivers/infiniband/hw/nes/nes.c > b/drivers/infiniband/hw/nes/nes.c > index a4e9269..d2884e7 100644 > --- a/drivers/infiniband/hw/nes/nes.c > +++ b/drivers/infiniband/hw/nes/nes.c > @@ -328,7 +328,7 @@ void nes_rem_ref(struct ib_qp *ibqp) > set_wqe_32bit_value(cqp_wqe->wqe_words, NES_CQP_WQE_ID_IDX, > nesqp->hwqp.qp_id); > u64temp = (u64)nesqp->nesqp_context_pbase; > set_wqe_64bit_value(cqp_wqe->wqe_words, > NES_CQP_QP_WQE_CONTEXT_LOW_IDX, u64temp); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > } > } > > diff --git a/drivers/infiniband/hw/nes/nes.h > b/drivers/infiniband/hw/nes/nes.h > index fe88bec..39bd897 100644 > --- a/drivers/infiniband/hw/nes/nes.h > +++ b/drivers/infiniband/hw/nes/nes.h > @@ -94,9 +94,6 @@ > > #define MAX_DPC_ITERATIONS 128 > > -#define NES_CQP_REQUEST_NO_DOORBELL_RING 0 > -#define NES_CQP_REQUEST_RING_DOORBELL 1 > - > #define NES_DRV_OPT_ENABLE_MPA_VER_0 0x00000001 > #define NES_DRV_OPT_DISABLE_MPA_CRC 0x00000002 > #define NES_DRV_OPT_DISABLE_FIRST_WRITE 0x00000004 > @@ -542,7 +539,7 @@ void nes_free_cqp_request(struct nes_device > *nesdev, > struct nes_cqp_request *cqp_request); > void nes_put_cqp_request(struct nes_device *nesdev, > struct nes_cqp_request *cqp_request); > -void nes_post_cqp_request(struct nes_device *, struct nes_cqp_request > *, int); > +void nes_post_cqp_request(struct nes_device *, struct nes_cqp_request > *); > int nes_arp_table(struct nes_device *, u32, u8 *, u32); > void nes_mh_fix(unsigned long); > void nes_clc(unsigned long); > diff --git a/drivers/infiniband/hw/nes/nes_hw.c > b/drivers/infiniband/hw/nes/nes_hw.c > index 0cea58b..59c2841 100644 > --- a/drivers/infiniband/hw/nes/nes_hw.c > +++ b/drivers/infiniband/hw/nes/nes_hw.c > @@ -3145,7 +3145,7 @@ int nes_manage_apbvt(struct nes_vnic *nesvnic, > u32 accel_local_port, > nes_debug(NES_DBG_QP, "Waiting for CQP completion for APBVT.\n"); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > if (add_port == NES_MANAGE_APBVT_ADD) > ret = wait_event_timeout(cqp_request->waitq, (cqp_request- > >request_done != 0), > @@ -3215,7 +3215,7 @@ void nes_manage_arp_cache(struct net_device > *netdev, unsigned char *mac_addr, > nesdev->cqp.sq_head, nesdev->cqp.sq_tail); > > atomic_set(&cqp_request->refcount, 1); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > } > > > @@ -3247,7 +3247,7 @@ void flush_wqes(struct nes_device *nesdev, struct > nes_qp *nesqp, > cpu_to_le32(NES_CQP_FLUSH_WQES | which_wq); > cqp_wqe->wqe_words[NES_CQP_WQE_ID_IDX] = cpu_to_le32(nesqp- > >hwqp.qp_id); > > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > if (wait_completion) { > /* Wait for CQP */ > diff --git a/drivers/infiniband/hw/nes/nes_utils.c > b/drivers/infiniband/hw/nes/nes_utils.c > index 8f519c8..fb8cbd7 100644 > --- a/drivers/infiniband/hw/nes/nes_utils.c > +++ b/drivers/infiniband/hw/nes/nes_utils.c > @@ -596,7 +596,7 @@ void nes_put_cqp_request(struct nes_device *nesdev, > * nes_post_cqp_request > */ > void nes_post_cqp_request(struct nes_device *nesdev, > - struct nes_cqp_request *cqp_request, int ring_doorbell) > + struct nes_cqp_request *cqp_request) > { > struct nes_hw_cqp_wqe *cqp_wqe; > unsigned long flags; > @@ -624,10 +624,9 @@ void nes_post_cqp_request(struct nes_device > *nesdev, > nesdev->cqp.sq_head, nesdev->cqp.sq_tail, > nesdev->cqp.sq_size, > cqp_request->waiting, atomic_read(&cqp_request- > >refcount)); > barrier(); > - if (ring_doorbell) { > - /* Ring doorbell (1 WQEs) */ > - nes_write32(nesdev->regs+NES_WQE_ALLOC, 0x01800000 | > nesdev->cqp.qp_id); > - } > + > + /* Ring doorbell (1 WQEs) */ > + nes_write32(nesdev->regs+NES_WQE_ALLOC, 0x01800000 | > nesdev->cqp.qp_id); > > barrier(); > } else { > diff --git a/drivers/infiniband/hw/nes/nes_verbs.c > b/drivers/infiniband/hw/nes/nes_verbs.c > index 464a98a..e3939d1 100644 > --- a/drivers/infiniband/hw/nes/nes_verbs.c > +++ b/drivers/infiniband/hw/nes/nes_verbs.c > @@ -118,7 +118,7 @@ static struct ib_mw *nes_alloc_mw(struct ib_pd > *ibpd) { > set_wqe_32bit_value(cqp_wqe->wqe_words, > NES_CQP_STAG_WQE_STAG_IDX, stag); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > ret = wait_event_timeout(cqp_request->waitq, (cqp_request- > >request_done != 0), > @@ -175,7 +175,7 @@ static int nes_dealloc_mw(struct ib_mw *ibmw) > set_wqe_32bit_value(cqp_wqe->wqe_words, > NES_CQP_STAG_WQE_STAG_IDX, ibmw->rkey); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > nes_debug(NES_DBG_MR, "Waiting for deallocate STag 0x%08X to > complete.\n", > @@ -477,7 +477,7 @@ static struct ib_fmr *nes_alloc_fmr(struct ib_pd > *ibpd, > (nesfmr->nesmr.pbls_used-1) : nesfmr- > >nesmr.pbls_used); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > ret = wait_event_timeout(cqp_request->waitq, (cqp_request- > >request_done != 0), > @@ -1417,7 +1417,7 @@ static struct ib_qp *nes_create_qp(struct ib_pd > *ibpd, > set_wqe_64bit_value(cqp_wqe->wqe_words, > NES_CQP_QP_WQE_CONTEXT_LOW_IDX, u64temp); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > nes_debug(NES_DBG_QP, "Waiting for create iWARP QP%u > to complete.\n", > @@ -1744,7 +1744,7 @@ static struct ib_cq *nes_create_cq(struct > ib_device *ibdev, int entries, > cpu_to_le32(((u32)((u64temp) >> 33)) & 0x7FFFFFFF); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > nes_debug(NES_DBG_CQ, "Waiting for create iWARP CQ%u to > complete.\n", > @@ -1841,7 +1841,7 @@ static int nes_destroy_cq(struct ib_cq *ib_cq) > (nescq->hw_cq.cq_number | ((u32)PCI_FUNC(nesdev->pcidev- > >devfn) << 16))); > nes_free_resource(nesadapter, nesadapter->allocated_cqs, nescq- > >hw_cq.cq_number); > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > nes_debug(NES_DBG_CQ, "Waiting for destroy iWARP CQ%u to > complete.\n", > @@ -1987,7 +1987,7 @@ static int nes_reg_mr(struct nes_device *nesdev, > struct nes_pd *nespd, > barrier(); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > ret = wait_event_timeout(cqp_request->waitq, (0 != cqp_request- > >request_done), > @@ -2638,7 +2638,7 @@ static int nes_dereg_mr(struct ib_mr *ib_mr) > set_wqe_32bit_value(cqp_wqe->wqe_words, > NES_CQP_STAG_WQE_STAG_IDX, ib_mr->rkey); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > nes_debug(NES_DBG_MR, "Waiting for deallocate STag 0x%08X > completed\n", ib_mr->rkey); > @@ -2809,7 +2809,7 @@ int nes_hw_modify_qp(struct nes_device *nesdev, > struct nes_qp *nesqp, > set_wqe_64bit_value(cqp_wqe->wqe_words, > NES_CQP_QP_WQE_CONTEXT_LOW_IDX, (u64)nesqp->nesqp_context_pbase); > > atomic_set(&cqp_request->refcount, 2); > - nes_post_cqp_request(nesdev, cqp_request, > NES_CQP_REQUEST_RING_DOORBELL); > + nes_post_cqp_request(nesdev, cqp_request); > > /* Wait for CQP */ > if (wait_completion) { > -- > 1.5.6 From rdreier at cisco.com Wed Jul 2 16:23:46 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 02 Jul 2008 16:23:46 -0700 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking needed for serialization of callbacks In-Reply-To: (Or Gerlitz's message of "Thu, 19 Jun 2008 17:07:01 +0300 (IDT)") References: Message-ID: am I correct in thinking that the first 2 patches have no dependencies on anything in another tree and could be merged now? From akpm at linux-foundation.org Wed Jul 2 12:40:02 2008 From: akpm at linux-foundation.org (Andrew Morton) Date: Wed, 2 Jul 2008 12:40:02 -0700 Subject: [ofa-general] Re: [PATCH 0 of 3] mmu notifier v18 for -mm In-Reply-To: References: Message-ID: <20080702124002.6d899621.akpm@linux-foundation.org> I merged the mmu-notifers patches ahead of these: # # gup # x86-implement-pte_special.patch mm-introduce-get_user_pages_fast.patch mm-introduce-get_user_pages_fast-fix.patch mm-introduce-get_user_pages_fast-checkpatch-fixes.patch x86-lockless-get_user_pages_fast.patch x86-lockless-get_user_pages_fast-checkpatch-fixes.patch x86-lockless-get_user_pages_fast-fix.patch x86-lockless-get_user_pages_fast-fix-2.patch x86-lockless-get_user_pages_fast-fix-2-fix-fix.patch x86-lockless-get_user_pages_fast-fix-warning.patch dio-use-get_user_pages_fast.patch splice-use-get_user_pages_fast.patch x86-support-1gb-hugepages-with-get_user_pages_lockless.patch # mm-readahead-scan-lockless.patch radix-tree-add-gang_lookup_slot-gang_lookup_slot_tag.patch #mm-speculative-page-references.patch: clameter saw bustage mm-speculative-page-references.patch mm-speculative-page-references-fix.patch mm-speculative-page-references-fix-fix.patch mm-speculative-page-references-hugh-fix3.patch mm-speculative-page-references-fix-migration_entry_wait-for-speculative-page-cache.patch mm-lockless-pagecache.patch mm-spinlock-tree_lock.patch powerpc-implement-pte_special.patch # vmscan-move-isolate_lru_page-to-vmscanc.patch vmscan-move-isolate_lru_page-to-vmscanc-fix.patch vmscan-use-an-indexed-array-for-lru-variables.patch swap-use-an-array-for-the-lru-pagevecs.patch vmscan-free-swap-space-on-swap-in-activation.patch define-page_file_cache-function.patch define-page_file_cache-function-fix.patch vmscan-split-lru-lists-into-anon-file-sets.patch vmscan-split-lru-lists-into-anon-file-sets-collect-lru-meminfo-statistics-from-correct-offset.patch vmscan-split-lru-lists-into-anon-file-sets-prevent-incorrect-oom-under-split_lru.patch vmscan-split-lru-lists-into-anon-file-sets-split_lru-fix-pagevec_move_tail-doesnt-treat-unevictable-page.patch vmscan-second-chance-replacement-for-anonymous-pages.patch vmscan-fix-pagecache-reclaim-referenced-bit-check.patch vmscan-add-newly-swapped-in-pages-to-the-inactive-list.patch more-aggressively-use-lumpy-reclaim.patch pageflag-helpers-for-configed-out-flags.patch unevictable-lru-infrastructure.patch unevictable-lru-infrastructure-fix.patch unevictable-lru-infrastructure-kconfig-fix.patch unevictable-lru-infrastructure-remove-redundant-page-mapping-check.patch unevictable-lru-page-statistics.patch unevictable-lru-page-statistics-fix-printk-in-show_free_areas.patch ramfs-and-ram-disk-pages-are-unevictable.patch ramfs-and-ram-disk-pages-are-unevictable-undo-the-brdc-part.patch shm_locked-pages-are-unevictable.patch mlock-mlocked-pages-are-unevictable.patch mlock-mlocked-pages-are-unevictable-fix.patch mlock-mlocked-pages-are-unevictable-fix-fix.patch mlock-mlocked-pages-are-unevictable-fix-3.patch mlock-mlocked-pages-are-unevictable-fix-4.patch mlock-mlocked-pages-are-unevictable-fix-fix-munlock-page-table-walk-now-requires-mm.patch mlock-downgrade-mmap-sem-while-populating-mlocked-regions.patch mmap-handle-mlocked-pages-during-map-remap-unmap.patch fix-double-unlock_page-in-2626-rc5-mm3-kernel-bug-at-mm-filemapc-575.patch mmap-handle-mlocked-pages-during-map-remap-unmap-cleanup.patch vmstat-mlocked-pages-statistics.patch vmstat-mlocked-pages-statistics-fix-incorrect-mlocked-field-of-proc-meminfo.patch swap-cull-unevictable-pages-in-fault-path.patch swap-cull-unevictable-pages-in-fault-path-fix.patch vmstat-unevictable-and-mlocked-pages-vm-events.patch vmscan-unevictable-lru-scan-sysctl.patch vmscan-unevictable-lru-scan-sysctl-nommu-fix.patch mlock-count-attempts-to-free-mlocked-page.patch doc-unevictable-lru-and-mlocked-pages-documentation.patch vmscam-kill-unused-lru-functions.patch with a view to merging the mmu-notifiers patches into 2.6.27-rc1. I expect that the above patches are 2.6.28 material. Consequently some of the above patches needed some rework which should be checked, please. In particular, do the above patches add new code which needs mmu-notifier treatment? Patches which actually required reject fixing were: unevictable-lru-infrastructure.patch mlock-mlocked-pages-are-unevictable.patch mmap-handle-mlocked-pages-during-map-remap-unmap.patch swap-cull-unevictable-pages-in-fault-path.patch but really the whole MM needs reviewing to check that the post-mmu-motifiers changes didn't break mmu notifiers. We have about three months in which to get that done. From panda at cse.ohio-state.edu Wed Jul 2 20:12:08 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Wed, 2 Jul 2008 23:12:08 -0400 (EDT) Subject: [ofa-general] Announcing the release of MVAPICH2 1.2RC1 Message-ID: The MVAPICH team is pleased to announce the release of MVAPICH2 1.2RC1 with the following NEW features: - Based on MPICH2 1.0.7 - Scalable and robust daemon-less job startup - Enhanced and robust mpirun_rsh framework (non-MPD-based) to provide scalable job launching on multi-thousand core clusters - Available for OpenFabrics (IB and iWARP) and uDAPL interfaces (including Solaris) - Checkpoint-restart with intra-node shared memory support - Allows best performance and scalability with fault-tolerance support - Enhancement to software installation - Full autoconf-based configuration - An application (mpiname) for querying the MVAPICH2 library version and configuration information - Enhanced processor affinity using PLPA for multi-core architectures - Allows user-defined flexible processor affinity - Enhanced scalability for RDMA-based direct one-sided communication with less communication resource - Shared memory optimized MPI_Bcast operation - Optimized and tuned MPI_Alltoall For downloading MVAPICH2 1.2RC1, associated user guide and accessing the SVN, please visit the following URL: http://mvapich.cse.ohio-state.edu All feedbacks, including bug reports and hints for performance tuning, patches and enhancements are welcome. Please post it to the mvapich-discuss mailing list. Thanks, The MVAPICH Team From ogerlitz at voltaire.com Wed Jul 2 22:48:14 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 03 Jul 2008 08:48:14 +0300 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking needed for serialization of callbacks In-Reply-To: References: Message-ID: <486C681E.6020102@voltaire.com> Roland Dreier wrote: > am I correct in thinking that the first 2 patches have no dependencies on anything in another tree and could be merged now? Oh, funny, the same thought crossed my mind yesterday... sure, you can merge them now, thanks. Or. From eli at dev.mellanox.co.il Wed Jul 2 23:41:34 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 3 Jul 2008 09:41:34 +0300 Subject: [ofa-general] [PATCH] IB/IPoIB: Fix change mtu when switching to UD mode In-Reply-To: <486B2F55.5020704@voltaire.com> References: <20080630160651.GA6024@mtls03> <4869CE9C.2050408@voltaire.com> <20080701071559.GE6024@mtls03> <486A1CB1.1020904@voltaire.com> <20080701134229.GC14384@mtls03> <486B2F55.5020704@voltaire.com> Message-ID: <20080703064134.GA15002@mtls03> On Wed, Jul 02, 2008 at 10:33:41AM +0300, Or Gerlitz wrote: > Eli Cohen wrote: >> The fact that I did not hit such an assert does not mean I may avoid >> taking the rtnl lock but it appears to me that the issue is not well >> undrestood. > Yes, linux networking internals are not easy to get... I suggest that you > send netdev an email with more concrete subject - eg "required rtnl locking > for netdev event notifiers..." which has larger chance to catch the eye of > the maintainers. > So after getting some (two) answers and looking searching "rtnl lock", I guess the correct way to go is to add rtnl lock/unlock. But I conclude that we have to extend the usage of this lock farther to wrap the modofications of the net device flags done, e.g. when changing modes. If will create two patches then. One which adds rtnl locking where necessary, and another one which uses dev_set_mtu() where needed. > From jackm at dev.mellanox.co.il Thu Jul 3 00:09:23 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 3 Jul 2008 10:09:23 +0300 Subject: [ofa-general] Re: [PATCH 8/11] core: XRC receive-only QPs In-Reply-To: <200807021804.11083.jackm@dev.mellanox.co.il> References: <200806231601.43706.jackm@dev.mellanox.co.il> <200807021804.11083.jackm@dev.mellanox.co.il> Message-ID: <200807031009.23638.jackm@dev.mellanox.co.il> On Wednesday 02 July 2008 18:04, Jack Morgenstein wrote: > On Tuesday 24 June 2008 00:53, Roland Dreier wrote: > > > @@ -769,6 +775,7 @@ struct ib_ucontext { > > > struct list_head srq_list; > > > struct list_head ah_list; > > > struct list_head xrc_domain_list; > > > + struct list_head xrc_reg_qp_list; > > > int closing; > > > }; > > > > Wouldn't it be cleaner to keep the like of recv QPs per xrcd? Then you > > wouldn't have to do stuff like: > > > > > + list_for_each_entry(tmp, &file->ucontext->xrc_reg_qp_list, list) > > > + if (cmd.xrcd_handle == tmp->domain_handle) { > > > > instead you could just do a list_empty() test. > > > Currently, there is no per-process object for xrc domains. I needed a list > of qp's registered ***per-process***, so that I could prevent closing a domain (for that process) > while that process still had registered QPs on that domain. This was especially true if > the current process was the last one using the domain -- so that the domain itself would be deleted > when close_domain was invoked. > > With the check in place, if the user tried to close the domain while there were QPs registered, > he gets an error. If the user does not close the domain, and does not unregister all the QPs > for that process, this gets done in ib_uverbs_cleanup_ucontext() (in the proper order), so there > are no resource leaks. > > I think that here we have "6 of one, half a dozen of the other": > If I register QPs per xrcd, I either have to create and manage an xrcd > user-space specific object (e.g., struct ib_uxrcd_object) -- instead of the generic ib_uobject, > or I need to save the process ID per QP in an xrcd list of QPs -- and then I would need to > check for equality of PIDs, rather than domain handles. > > I think the way things are now is relatively simpler. On second thought, I'll look at this again for a day or two. It is a medium-complexity code change, which needs to be thoroughly tested out. I'll need to define an ib_uxrcd_object, and place the xrc_reg_qp_list in that uobject. When cleaning up the process, I'll just need to unregister all the xrc_rcv_qp's in a loop within the xrc domain cleanup loop (i.e. list_for_each_entry_safe(uobj, tmp, &context->xrc_domain_list, list) { struct ib_xrcd *xrcd = uobj->object; ===> xrc_rcv_qp list cleanup here, in a "for each" loop idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); ib_uverbs_dealloc_xrcd(file->device->ib_dev, xrcd); kfree(uobj); } mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); ) When registering/unregistering xrc_rcv_qp's, I need to manage the qp list within the ib_uxrcd_object. I do agree that this would be logically cleaner, since this way the receive qp's per process xrc domain are contained in the process' xrc domain object. - Jack From jean-vincent.ficet at bull.net Thu Jul 3 01:41:33 2008 From: jean-vincent.ficet at bull.net (Vincent Ficet) Date: Thu, 03 Jul 2008 10:41:33 +0200 Subject: [ofa-general] Issues with osm_sm_state_mgr_check_legality() Message-ID: <486C90BD.5040807@bull.net> Hello, In opensm/opensm/osm_sm_state_mgr.c:osm_sm_state_mgr_check_legality(), I noticed that the only valid signals that the SM can get while in the IB_SMINFO_STATE MASTER are: - OSM_SM_SIGNAL_HANDOVER - OSM_SM_SIGNAL_HANDOVER_SENT However, in opensm/opensm/osm_sm_state_mgr.c:osm_sm_state_mgr_process(), we see that the following signals can be received in the master state: - OSM_SM_SIGNAL_POLLING_TIMEOUT - OSM_SM_SIGNAL_HANDOVER - OSM_SM_SIGNAL_HANDOVER_SENT - OSM_SM_SIGNAL_WAIT_FOR_HANDOVER - OSM_SM_SIGNAL_DISCOVER Is this a bug, or am I missing anything here ? Thanks for your help, Vincent From eli at mellanox.co.il Thu Jul 3 01:52:10 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Thu, 3 Jul 2008 11:52:10 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: Use rtnl lock/unlock when changing device flags Message-ID: <20080703085210.GA8511@mtls03> Use of this lock is required to synchronize changes to the netdvice's data structs. Also moved the call to ipoib_flush_paths() after the modification of the netdevice flags. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++++++-- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 5 ++++- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 97e67d3..f72e37d 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1384,7 +1384,9 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); + rtnl_lock(); dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); + rtnl_unlock(); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; ipoib_flush_paths(dev); @@ -1393,14 +1395,16 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, if (!strcmp(buf, "datagram\n")) { clear_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); - dev->mtu = min(priv->mcast_mtu, dev->mtu); - ipoib_flush_paths(dev); + rtnl_lock(); if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; if (priv->hca_caps & IB_DEVICE_UD_TSO) dev->features |= NETIF_F_TSO; } + dev->mtu = min(priv->mcast_mtu, dev->mtu); + rtnl_unlock(); + ipoib_flush_paths(dev); return count; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 3f663fb..189fe1f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -575,8 +575,11 @@ void ipoib_mcast_join_task(struct work_struct *work) priv->mcast_mtu = IPOIB_UD_MTU(ib_mtu_enum_to_int(priv->broadcast->mcmember.mtu)); - if (!ipoib_cm_admin_enabled(dev)) + if (!ipoib_cm_admin_enabled(dev)) { + rtnl_lock(); dev->mtu = min(priv->mcast_mtu, priv->admin_mtu); + rtnl_unlock(); + } ipoib_dbg_mcast(priv, "successfully joined all multicast groups\n"); -- 1.5.6 From eli at mellanox.co.il Thu Jul 3 01:52:45 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Thu, 3 Jul 2008 11:52:45 +0300 Subject: [ofa-general] [PATCH] Subject: [PATCH] IB/IPoIB: Fix change of mtu to use kernel API Message-ID: <20080703085245.GA8551@mtls03> When the driver sets the MTU of the net device, it should make use of dev_set_mtu() instead of directly setting the mtu field of struct netdevice. Failure to do so will result in functions registered to be called upon MTU change will not get called (this is done through call_netdevice_notifiers()). We have seen problems with performance degredation upon changing the MTU with ofed which has a patch to set the MTU when changing to CM mode to its maximum allowed value. Using dev_set_mtu() solved that problem. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index f72e37d..99a4daf 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1402,7 +1402,7 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, if (priv->hca_caps & IB_DEVICE_UD_TSO) dev->features |= NETIF_F_TSO; } - dev->mtu = min(priv->mcast_mtu, dev->mtu); + dev_set_mtu(dev, min(priv->mcast_mtu, dev->mtu)); rtnl_unlock(); ipoib_flush_paths(dev); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 189fe1f..f349079 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -577,7 +577,7 @@ void ipoib_mcast_join_task(struct work_struct *work) if (!ipoib_cm_admin_enabled(dev)) { rtnl_lock(); - dev->mtu = min(priv->mcast_mtu, priv->admin_mtu); + dev_set_mtu(dev, min(priv->mcast_mtu, priv->admin_mtu)); rtnl_unlock(); } -- 1.5.6 From eli at mellanox.co.il Thu Jul 3 01:53:29 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Thu, 3 Jul 2008 11:53:29 +0300 Subject: [ofa-general] [PATCH] Subject: [PATCH] IB/ipoib: set max CM MTU when moving to CM mode Message-ID: <20080703085329.GA8583@mtls03> This will relieve the user from the need to restore CM mode MTU every time he switchs from UD to CM mode. With the current code, if the user fails to reset the MTU to a decent higher value, he will get poor performance. Signed-off-by: Eli Cohen --- --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 99a4daf..eba92e7 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1384,8 +1384,12 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); + if (ipoib_cm_max_mtu(dev) > priv->mcast_mtu) + ipoib_warn(priv, "mtu > %d will cause multicast packet drops.\n", + priv->mcast_mtu); rtnl_lock(); dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); + dev_set_mtu(dev, ipoib_cm_max_mtu(dev)); rtnl_unlock(); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; -- 1.5.6 From vlad at lists.openfabrics.org Thu Jul 3 02:38:22 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Thu, 3 Jul 2008 02:38:22 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080703-0200 daily build status Message-ID: <20080703093822.949BCE60B08@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c: At top level: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: error: unknown field 'change_queue_depth' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080703-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From jason.orr2 at mail.dcu.ie Thu Jul 3 02:47:20 2008 From: jason.orr2 at mail.dcu.ie (Irish Lottery News Center) Date: Thu, 3 Jul 2008 02:47:20 -0700 Subject: [ofa-general] Final Notification/ Confirm Your Winning Amount Message-ID: <4858FDD10001425F@hawk.dcu.ie> Ref: UK/9420X2/68 Batch: 074/05/ZY369 This is to inform you that you have been selected for a cash prize of £750,000.00(GBP)in cash credited to file KTU/9023118308/03 held on the month of June 2008. To claim your prize,Contact Mr Derek White with the following informations. Name,Address,Age,Occupation,Country Email:customersupportunit.irn at hotmail.com Mr Clark Johnson From psbradynn at unipath.com Thu Jul 3 03:07:28 2008 From: psbradynn at unipath.com (Downloads erlaubt) Date: Thu, 3 Jul 2008 11:07:28 +0100 Subject: [ofa-general] Diese Preise Werden Sie Ueberzeugen, ganz sicher. Message-ID: <775625526.81545706214522@unipath.com> Die Software auf allen europaischen Sprachen, fuer Windows und Macintosh vorherbestimmt. Die koennen Sie momentan bekommen. Nur bezahlen und auslasten. Hier prasentiert sind nicht teuere, aber echte und voellige Produkte der Software. Nur die neueste und beste Software http://waqwosoft.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at dev.mellanox.co.il Thu Jul 3 03:45:50 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 03 Jul 2008 13:45:50 +0300 Subject: [ofa-general] Announcing the release of MVAPICH2 1.2RC1 In-Reply-To: References: Message-ID: <486CADDE.9040405@mellanox.co.il> Dhabaleswar Panda wrote: > The MVAPICH team is pleased to announce the release of MVAPICH2 1.2RC1 > with the following NEW features: > > > Thanks for the update. When do you plan to submit the new tarball with this release for OFED 1.4? Tziporet From jason.orr2 at mail.dcu.ie Thu Jul 3 03:49:39 2008 From: jason.orr2 at mail.dcu.ie (Irish Lottery News Center) Date: Thu, 3 Jul 2008 03:49:39 -0700 Subject: [ofa-general] Final Notification/ Confirm Your Winning Amount Message-ID: <4858FDD10001470F@hawk.dcu.ie> Ref: UK/9420X2/68 Batch: 074/05/ZY369 This is to inform you that you have been selected for a cash prize of £750,000.00(GBP)in cash credited to file KTU/9023118308/03 held on the month of June 2008. To claim your prize,Contact Mr Derek White with the following informations. Name,Address,Age,Occupation,Country Email:customersupportunit.irn at hotmail.com Mr Clark Johnson From alexanmg at slu.edu Thu Jul 3 05:19:40 2008 From: alexanmg at slu.edu (UK NATIONAL LOTTERY) Date: Thu, 03 Jul 2008 07:19:40 -0500 Subject: [ofa-general] Confirm Your Winning Message-ID: <486cc3dc.f.42ab.1209769647@slu.edu> You won the sum of £2,000000.00 GBP, Pounds sterling from the ballot lottery international program, Promotion; you are here by advised to get back to us,to claim your prize. Contact our claims agent for validation: Barrister Kelly Moore. Email:kellymooreclaims at live.com Tel :+( 44) 70457 09556 1. Full name: 2. Home Address: 3. Age: 4. Sex: 5. Phone Number: 6. Country of Residence: 7. Occupation: Mrs. Dianne Thompson. From panda at cse.ohio-state.edu Thu Jul 3 05:45:47 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Thu, 3 Jul 2008 08:45:47 -0400 (EDT) Subject: [ofa-general] Announcing the release of MVAPICH2 1.2RC1 In-Reply-To: <486CADDE.9040405@mellanox.co.il> Message-ID: Tziporet, > > The MVAPICH team is pleased to announce the release of MVAPICH2 1.2RC1 > > with the following NEW features: > > > > > > > Thanks for the update. > When do you plan to submit the new tarball with this release for OFED 1.4? We are carrying out some extensive testing now. As soon as this is over, we will submit a version for OFED 1.4 testing. Mostly, it should be during the next 1-2 weeks. Thanks, DK > Tziporet > From ogerlitz at voltaire.com Thu Jul 3 06:34:59 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 03 Jul 2008 16:34:59 +0300 Subject: [ofa-general] Re: [PATCH v4 for-2.6.27] IPOIB: add LRO support. In-Reply-To: <20080702140351.GD13098@mtls03> References: <4867D58C.5030404@gmail.com> <486A1E29.6080201@voltaire.com> <486B8189.3030507@voltaire.com> <20080702140351.GD13098@mtls03> Message-ID: <486CD583.5030103@voltaire.com> Eli Cohen wrote: > the 64K comes from the fact that an IP datagram's max length is 64K. And since LRO involves generating at the receiver a hand made IP datagram, the max length is 64K. The .max_aggr field comes to say "Max number of LRO packets to be aggregated" and not "max number of k bytes to aggregate. As you say, the packets are aggregated into a legal IP packet which is pushed up to the stack, whose size can not exceed 64k and as such the __lro_proc_skb code at net/ipv4/inet_lro.c does > if ((lro_desc->pkt_aggr_cnt >= lro_mgr->max_aggr) || > lro_desc->parent->len > (0xFFFF - lro_mgr->dev->mtu)) > lro_flush(lro_mgr, lro_desc); so in the case of a 64K message sent over 2044 mtu, .max_aggr of 32 or 33 would serve well as large-receive-offload which is a software-based mirror mechanism for LSO. For the case of small --messages-- eg of 1K, I don't see why we want to aggregate more then 32 of them, is it just because this code appears in other drivers? Or. From jason.orr2 at mail.dcu.ie Thu Jul 3 06:49:09 2008 From: jason.orr2 at mail.dcu.ie (Irish Lottery News Center) Date: Thu, 3 Jul 2008 06:49:09 -0700 Subject: [ofa-general] Confirm Your Winning Amount Message-ID: <4858FDD100015510@hawk.dcu.ie> Ref: UK/9420X2/68 Batch: 074/05/ZY369 This is to inform you that you have been selected for a cash prize of £750,000.00(GBP)in cash credited to file KTU/9023118308/03 held on the month of June 2008. To claim your prize,Contact Mr Derek White with the following informations. Name,Address,Age,Occupation,Country Email:irn.customerunit at live.com Mr Clark Johnson irn.customerunit at live.com From eli at dev.mellanox.co.il Thu Jul 3 07:51:49 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 3 Jul 2008 17:51:49 +0300 Subject: [ofa-general] Re: [PATCH v4 for-2.6.27] IPOIB: add LRO support. In-Reply-To: <486CD583.5030103@voltaire.com> References: <4867D58C.5030404@gmail.com> <486A1E29.6080201@voltaire.com> <486B8189.3030507@voltaire.com> <20080702140351.GD13098@mtls03> <486CD583.5030103@voltaire.com> Message-ID: <20080703145149.GA8615@mtls03> On Thu, Jul 03, 2008 at 04:34:59PM +0300, Or Gerlitz wrote: > > For the case of small --messages-- eg of 1K, I don't see why we want to > aggregate more then 32 of them, is it just because this code appears in > other drivers? > We do want to aggregate as much as possible since it means that less SKBs are passed up the networking stack. Of course we can't exceed 64K and we're bound to a single NAPI poll session but I still think 64 is not too large a number. From ctung at NetEffect.com Thu Jul 3 08:08:09 2008 From: ctung at NetEffect.com (Chien Tung) Date: Thu, 3 Jul 2008 10:08:09 -0500 Subject: [ofa-general] RE: [PATCH] RDMA/nes: Remove write-only fields of struct nes_vnic In-Reply-To: References: Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC0829CCAC@venom2> > From: Roland Dreier [mailto:rdreier at cisco.com] > Sent: Tuesday, July 01, 2008 11:07 PM > struct nes_vnic.get_cqp_request and .post_cqp_request are > assigned but never used. Remove them. > > Signed-off-by: Roland Dreier > --- > And another one... let me know if there is some plan to use > these fields in the future and I'll drop this. Roland, We do have plans for these fields, please do not remove them. Thanks, Chien From rdreier at cisco.com Thu Jul 3 08:12:29 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 08:12:29 -0700 Subject: [ofa-general] Re: [PATCH] RDMA/nes: Remove write-only fields of struct nes_vnic In-Reply-To: <5E701717F2B2ED4EA60F87C8AA57B7CC0829CCAC@venom2> (Chien Tung's message of "Thu, 3 Jul 2008 10:08:09 -0500") References: <5E701717F2B2ED4EA60F87C8AA57B7CC0829CCAC@venom2> Message-ID: > We do have plans for these fields, please do not remove them. OK, I'll drop this patch. From pscabaret at aurecan.e.telefonica.net Thu Jul 3 08:31:06 2008 From: pscabaret at aurecan.e.telefonica.net (Clifford York) Date: Thu, 3 Jul 2008 12:31:06 -0300 Subject: [ofa-general] Visit our official site Message-ID: <000e01c8dd08$a9f4d1d0$00e1bb0c@comercial> THIS will keep it up all night long. we will fulfill your urgent medical needs. Here! ex-brother evaporable fahnestock farrieries englischen execufirst fBmailhelp excalbians fashioneth exiguously fLlastcomm fBrfpasswd -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Thu Jul 3 08:55:45 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 03 Jul 2008 18:55:45 +0300 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> References: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> Message-ID: <486CF681.6050206@voltaire.com> Ron Livne wrote: > We would like to start working on user space verbs for blocking multicast packets as was discussed in a previous thread. I would like to implement it simply by adding a new verb (kind of extended create_qp). I think this implementation will have similar characteristics with the XRC. Do you have any suggestions or preferences? Hi Roland, Can you comment on that? basically the thing is that we want to expose some functionality to user space which already exists in the kernel, such as block multicast loopback and CQ moderation. We understand that Jack is working now on the XRC merge whose volume is much more bigger, and there's no intention to step or disturb on this effort, just want you to have in mind these smaller changes and ask what's your thinking on the way to do them. It seems to me more constructive then to just send patches on top of Jack ones... Or From redmond at ingman.com Thu Jul 3 08:35:49 2008 From: redmond at ingman.com (derry becket) Date: Thu, 03 Jul 2008 15:35:49 +0000 Subject: [ofa-general] New Great discount stores Message-ID: <000a01c8dd31$06332bd5$af922b88@ghyrpa> http://www.topsuperb.com/ Accessories that will surprise her - get it now at a discount -------------- next part -------------- An HTML attachment was scrubbed... URL: From pw at osc.edu Thu Jul 3 11:27:45 2008 From: pw at osc.edu (Pete Wyckoff) Date: Thu, 3 Jul 2008 14:27:45 -0400 Subject: [ofa-general] [PATCH] ib cm conn sent destroy Message-ID: <20080703182745.GB1118@osc.edu> Admittedly, I'm in this situation due to working on broken device code, but I think the patch below addresses a valid problem. Against 2.6.26-rc8. Consider a very slow link and an impatient client. It does rdma_connect(), then times out at the application layer while the IW_CM_STATE is still CONN_SENT. It calls rdma_reject() and finally rdma_destroy_id(). That leads to the BUG() in destroy_cm_id(). Note the call to rdma_reject() is necessary to avoid hanging at the top of destroy_cm_id(), waiting for the IWCM_F_CONNECT_WAIT bit to be cleared. -- Pete ---------------------------------------------------------------- An application may call rdma_destroy_id() when in the CONN_SENT state. This should also send a reject to the provider, just like for the CONN_RECV case. It currently BUG()s without this patch. Signed-off-by: Pete Wyckoff --- drivers/infiniband/core/iwcm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index 81c9195..acffdea 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -354,6 +354,7 @@ static void destroy_cm_id(struct iw_cm_id *cm_id) case IW_CM_STATE_CLOSING: cm_id_priv->state = IW_CM_STATE_DESTROYING; break; + case IW_CM_STATE_CONN_SENT: case IW_CM_STATE_CONN_RECV: /* * App called destroy before/without calling accept after @@ -363,7 +364,6 @@ static void destroy_cm_id(struct iw_cm_id *cm_id) */ cm_id_priv->state = IW_CM_STATE_DESTROYING; break; - case IW_CM_STATE_CONN_SENT: case IW_CM_STATE_DESTROYING: default: BUG(); -- 1.5.5.1 From swise at opengridcomputing.com Thu Jul 3 11:39:25 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 03 Jul 2008 13:39:25 -0500 Subject: [ofa-general] Re: [PATCH] ib cm conn sent destroy In-Reply-To: <20080703182745.GB1118@osc.edu> References: <20080703182745.GB1118@osc.edu> Message-ID: <486D1CDD.3080107@opengridcomputing.com> Pete Wyckoff wrote: > Admittedly, I'm in this situation due to working on broken device > code, but I think the patch below addresses a valid problem. > Against 2.6.26-rc8. > > Consider a very slow link and an impatient client. It does > rdma_connect(), then times out at the application layer while the > IW_CM_STATE is still CONN_SENT. It calls rdma_reject() and finally > rdma_destroy_id(). That leads to the BUG() in destroy_cm_id(). > > Note the call to rdma_reject() is necessary to avoid hanging at the > top of destroy_cm_id(), waiting for the IWCM_F_CONNECT_WAIT bit to > be cleared. > > This isn't the intended usage model. rdma_accept() and rdma_reject() are only valid on connect request cm_ids passed up in the CONNECT_REQUEST event on the server side... (I'm not saying something isn't broken however :) > -- Pete > > ---------------------------------------------------------------- > > An application may call rdma_destroy_id() when in the CONN_SENT > state. This should also send a reject to the provider, just like > for the CONN_RECV case. It currently BUG()s without this patch. > > Signed-off-by: Pete Wyckoff > --- > drivers/infiniband/core/iwcm.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c > index 81c9195..acffdea 100644 > --- a/drivers/infiniband/core/iwcm.c > +++ b/drivers/infiniband/core/iwcm.c > @@ -354,6 +354,7 @@ static void destroy_cm_id(struct iw_cm_id *cm_id) > case IW_CM_STATE_CLOSING: > cm_id_priv->state = IW_CM_STATE_DESTROYING; > break; > + case IW_CM_STATE_CONN_SENT: > case IW_CM_STATE_CONN_RECV: > /* > * App called destroy before/without calling accept after > @@ -363,7 +364,6 @@ static void destroy_cm_id(struct iw_cm_id *cm_id) > */ > cm_id_priv->state = IW_CM_STATE_DESTROYING; > break; > - case IW_CM_STATE_CONN_SENT: > case IW_CM_STATE_DESTROYING: > default: > BUG(); > From pw at osc.edu Thu Jul 3 12:25:56 2008 From: pw at osc.edu (Pete Wyckoff) Date: Thu, 3 Jul 2008 15:25:56 -0400 Subject: [ofa-general] Re: [PATCH] ib cm conn sent destroy In-Reply-To: <486D1CDD.3080107@opengridcomputing.com> References: <20080703182745.GB1118@osc.edu> <486D1CDD.3080107@opengridcomputing.com> Message-ID: <20080703192556.GA1179@osc.edu> swise at opengridcomputing.com wrote on Thu, 03 Jul 2008 13:39 -0500: > Pete Wyckoff wrote: >> Admittedly, I'm in this situation due to working on broken device >> code, but I think the patch below addresses a valid problem. >> Against 2.6.26-rc8. >> >> Consider a very slow link and an impatient client. It does >> rdma_connect(), then times out at the application layer while the >> IW_CM_STATE is still CONN_SENT. It calls rdma_reject() and finally >> rdma_destroy_id(). That leads to the BUG() in destroy_cm_id(). >> >> Note the call to rdma_reject() is necessary to avoid hanging at the >> top of destroy_cm_id(), waiting for the IWCM_F_CONNECT_WAIT bit to >> be cleared. >> >> > This isn't the intended usage model. rdma_accept() and rdma_reject() > are only valid on connect request cm_ids passed up in the > CONNECT_REQUEST event on the server side... > > (I'm not saying something isn't broken however :) That makes sense. I couldn't find anyway to get that bit IWCM_F_CONNECT_WAIT cleared (without massive layering violations). Is there anything like rdma_give_up_on_the_connect() that works on the client side? Am I expected just to wait for the transport layer to time out the connect? I could imagine a Ctrl-C situation where immediate client-side cancellation is desired. -- Pete From swise at opengridcomputing.com Thu Jul 3 12:32:18 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 03 Jul 2008 14:32:18 -0500 Subject: [ofa-general] Re: [PATCH] ib cm conn sent destroy In-Reply-To: <20080703192556.GA1179@osc.edu> References: <20080703182745.GB1118@osc.edu> <486D1CDD.3080107@opengridcomputing.com> <20080703192556.GA1179@osc.edu> Message-ID: <486D2942.1060409@opengridcomputing.com> >>> >>> >> This isn't the intended usage model. rdma_accept() and rdma_reject() >> are only valid on connect request cm_ids passed up in the >> CONNECT_REQUEST event on the server side... >> >> (I'm not saying something isn't broken however :) >> > > That makes sense. I couldn't find anyway to get that bit > IWCM_F_CONNECT_WAIT cleared (without massive layering violations). > Is there anything like rdma_give_up_on_the_connect() that works > on the client side? Am I expected just to wait for the transport > layer to time out the connect? I could imagine a Ctrl-C situation > where immediate client-side cancellation is desired. > > -- Pete > Currently, no. The IWCM must wait until the iwarp provider finishes the connect attempt. That's the current design. Its busted IMO... From sean.hefty at intel.com Thu Jul 3 13:03:55 2008 From: sean.hefty at intel.com (Hefty, Sean) Date: Thu, 3 Jul 2008 13:03:55 -0700 Subject: [ofa-general] Re: [PATCH] iw cm conn sent destroy In-Reply-To: <20080703192556.GA1179@osc.edu> References: <20080703182745.GB1118@osc.edu><486D1CDD.3080107@opengridcomputing.com> <20080703192556.GA1179@osc.edu> Message-ID: >That makes sense. I couldn't find anyway to get that bit >IWCM_F_CONNECT_WAIT cleared (without massive layering violations). >Is there anything like rdma_give_up_on_the_connect() that works >on the client side? Am I expected just to wait for the transport >layer to time out the connect? I could imagine a Ctrl-C situation >where immediate client-side cancellation is desired. >From the perspective of the rdma_cm interface, rdma_destroy_id() is intended to provide this functionality. - Sean From swise at opengridcomputing.com Thu Jul 3 13:16:02 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 03 Jul 2008 15:16:02 -0500 Subject: [ofa-general] [PATCH for-2.6.27] RDMA/cxgb3: Memory window rkey not being set. Message-ID: <20080703201602.17876.8085.stgit@dell3.ogc.int> From: Steve Wise iwch_alloc_mw() was not setting the rkey field in the newly allocated struct ib_mw. Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/iwch_provider.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index d3c99d9..ba6eb83 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -748,6 +748,7 @@ static struct ib_mw *iwch_alloc_mw(struct ib_pd *pd) mhp->attr.type = TPT_MW; mhp->attr.stag = stag; mmid = (stag) >> 8; + mhp->ibmw.rkey = stag; insert_handle(rhp, &rhp->mmidr, mhp, mmid); PDBG("%s mmid 0x%x mhp %p stag 0x%x\n", __func__, mmid, mhp, stag); return &(mhp->ibmw); From pw at osc.edu Thu Jul 3 13:37:54 2008 From: pw at osc.edu (Pete Wyckoff) Date: Thu, 3 Jul 2008 16:37:54 -0400 Subject: [ofa-general] Re: [PATCH] iw cm conn sent destroy In-Reply-To: References: <20080703192556.GA1179@osc.edu> Message-ID: <20080703203754.GC1304@osc.edu> sean.hefty at intel.com wrote on Thu, 03 Jul 2008 13:03 -0700: > >That makes sense. I couldn't find anyway to get that bit > >IWCM_F_CONNECT_WAIT cleared (without massive layering violations). > >Is there anything like rdma_give_up_on_the_connect() that works > >on the client side? Am I expected just to wait for the transport > >layer to time out the connect? I could imagine a Ctrl-C situation > >where immediate client-side cancellation is desired. > > From the perspective of the rdma_cm interface, rdma_destroy_id() is > intended to provide this functionality. That is a sensible approach. The implementation in iwcm.c just passively waits for a connection to time out before allowing rdma_destroy_id() to complete. Maybe I should look at adding a case in cma_cancel_operation() for CMA_CONNECT that calls into a device-specific function that can actively quit the connection. Or should this be a special case in iwcm's destroy_cm_id()? Where a CONN_SENT state will bypass the check on IWCM_F_CONNECT_WAIT, and call into a device function that tries to cancel the connection attepmt first? -- Pete From swise at opengridcomputing.com Thu Jul 3 13:44:17 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 03 Jul 2008 15:44:17 -0500 Subject: [ofa-general] Re: [PATCH] iw cm conn sent destroy In-Reply-To: <20080703203754.GC1304@osc.edu> References: <20080703192556.GA1179@osc.edu> <20080703203754.GC1304@osc.edu> Message-ID: <486D3A21.2030205@opengridcomputing.com> Pete Wyckoff wrote: > sean.hefty at intel.com wrote on Thu, 03 Jul 2008 13:03 -0700: > >>> That makes sense. I couldn't find anyway to get that bit >>> IWCM_F_CONNECT_WAIT cleared (without massive layering violations). >>> Is there anything like rdma_give_up_on_the_connect() that works >>> on the client side? Am I expected just to wait for the transport >>> layer to time out the connect? I could imagine a Ctrl-C situation >>> where immediate client-side cancellation is desired. >>> >> From the perspective of the rdma_cm interface, rdma_destroy_id() is >> intended to provide this functionality. >> > > That is a sensible approach. The implementation in iwcm.c just > passively waits for a connection to time out before allowing > rdma_destroy_id() to complete. Maybe I should look at adding a case > in cma_cancel_operation() for CMA_CONNECT that calls into a > device-specific function that can actively quit the connection. > > Or should this be a special case in iwcm's destroy_cm_id()? Where > a CONN_SENT state will bypass the check on IWCM_F_CONNECT_WAIT, and > call into a device function that tries to cancel the connection > attepmt first? > > -- Pete > If we define a method that every iwarp provider supports to cancel a connect attempt, then we can do away with the IWCM_F_CONNECT_WAIT bit altogether and always cancel the operation instead. For cxgb3, the connect request to the HW really cannot be canceled. So the driver would have to keep its endpoint struct around until the HW finally posts the connect attempt results, but mark the endpoint as "no cm_id attached". The if the connection succeeds, the driver will immediately reset it I guess... Steve. From rdreier at cisco.com Thu Jul 3 14:14:48 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 14:14:48 -0700 Subject: [ofa-general] Re: [PATCH 8/11] core: XRC receive-only QPs In-Reply-To: <200807031009.23638.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Thu, 3 Jul 2008 10:09:23 +0300") References: <200806231601.43706.jackm@dev.mellanox.co.il> <200807021804.11083.jackm@dev.mellanox.co.il> <200807031009.23638.jackm@dev.mellanox.co.il> Message-ID: > I'll need to define an ib_uxrcd_object I think that's the right thing to do. It keeps the XRC code structured in a similar way to the other uverbs objects. From rdreier at cisco.com Thu Jul 3 14:15:52 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 14:15:52 -0700 Subject: [ofa-general] Re: [PATCH] - [resend] Corrects a race in ipoib_cm_post_receive_nonsrq() In-Reply-To: (Pradeep Satyanarayana's message of "Wed, 2 Jul 2008 06:10:13 -0700") References: Message-ID: > Thanks for writing up this patch. I have been extremely busy dealing with > some hot issues and hence the delayed response. I did see Shirley's > response and yes it looks okay to me too. In the same vein that we have > been distinguishing the nonsrq functions (from the generic or srq ones), > would you consider renaming ipoib_cm_init_rx_wr() to something with nonsrq > suffix? Thanks! That would make sense but ipoib_cm_init_rx_wr() is also called from ipoib_cm_dev_init() even when an SRQ is available. So it's not really a "nonsrq" function. - R. From rdreier at cisco.com Thu Jul 3 14:18:43 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 14:18:43 -0700 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <486CF681.6050206@voltaire.com> (Or Gerlitz's message of "Thu, 03 Jul 2008 18:55:45 +0300") References: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> <486CF681.6050206@voltaire.com> Message-ID: > > We would like to start working on user space verbs for blocking multicast packets as was discussed in a previous thread. I would like to implement it simply by adding a new verb (kind of extended create_qp). I think this implementation will have similar characteristics with the XRC. Do you have any suggestions or preferences? > Can you comment on that? basically the thing is that we want to expose > some functionality to user space which already exists in the kernel, > such as block multicast loopback and CQ moderation. Hmm, I missed the original email. Anyway I have no idea how to do the block multicast feature really. I thought there was some idea that userspace would pass QP creation flags into the kernel? Was there some reason that XRC wanted to use QP creation flags too? - R. From cadfred at copper.net Thu Jul 3 06:09:30 2008 From: cadfred at copper.net (cadfred at copper.net) Date: Thu, 03 Jul 2008 17:09:30 +0400 Subject: [ofa-general] Federal grants available Message-ID: Press Release general at lists.openfabrics.org The American Grants and Loans Book is now available. This publication contains valuable information with more than 1800 financial programs, subsidies, scholarships, grants and loans offered by the United States federal government. It also includes over 700 financing programs put forth by various Foundations and Associations across the United States. Businesses, students, individuals, municipalities, government departments, institutions, foundations and associations will find a wealth of information that will help them with their new ventures or existing projects. What you get: -Description of Grant available -Url to government website -Full mailing address -Phone and fax number The Canadian Subsidy Directory is also available for Canada. CD version: $69.95 Printed version: $149.95 To order please call: 819-322-7533 If you do not wish to receive communication from us in the future please write "agl" in the subject line to: unsub2 at hotpop.com Canada Books 833 Boise de la Riviere Prevost, Qc Canada From rdreier at cisco.com Thu Jul 3 14:20:38 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 14:20:38 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27] RDMA/cxgb3: Memory window rkey not being set. In-Reply-To: <20080703201602.17876.8085.stgit@dell3.ogc.int> (Steve Wise's message of "Thu, 03 Jul 2008 15:16:02 -0500") References: <20080703201602.17876.8085.stgit@dell3.ogc.int> Message-ID: thanks, applied From rdreier at cisco.com Thu Jul 3 14:31:08 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 14:31:08 -0700 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: (Roland Dreier's message of "Thu, 03 Jul 2008 14:18:43 -0700") References: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> <486CF681.6050206@voltaire.com> Message-ID: > Anyway I have no idea how to do the block multicast feature really. I > thought there was some idea that userspace would pass QP creation flags > into the kernel? > > Was there some reason that XRC wanted to use QP creation flags too? And one thing you could do would be to look at the XRC patches and see if there are any aspects that might have an effect on your block loopback feature. If there is something that you want to generalize/share, now is the time to figure it out (before we merge the code and have a harder time changing the ABI). And so on. From jon at opengridcomputing.com Thu Jul 3 14:34:12 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Thu, 3 Jul 2008 16:34:12 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support Message-ID: <20080703213412.GB32285@opengridcomputing.com> This patch adds support for running RDS over iWARP adapters. It requires the fast_reg_mr patches and Olaf's RDS patches (for the flow control enablement) to apply cleanly and build. It also includes the patch I sent out earlier to have different connection ports for IB, TCP, and IWARP. It is a "work in progress", but I would like for people to take a look and let me know what they think. I was able to run rds-ping and rds-stress for lengths of time without problems (with the exception of the rds-stress "-D" flag, which enabled RDMA support). With fast_reg_mr support, I was able to get RDMA to work at small intervals but there are still bugs to be worked out. The bulk of the code is a copy of the RDS IB with the changes necessary to get iWARP working. Thus there may be stale comments and redundant code. I will audit the code to remove the stale comments, and remove duplicated and unmodified code (but feel free to point out any that you see). The logic in the RDS connection setup to determine which method to use, requires a new function in the core IB logic to translate a given IP Address to the IB/iWARP device associated with it. The only place this can be determined is in the pre-existing structs in cma.c. Because the RDS iWARP get_mr function needs to know the QP associated with the mr, more data is now needed to be passed around. Luckily, the QP can be determined based on the rds_sock (which also contains the ip_addr). By using this struct, the addition of another function param to get_mr is not necessary. However, this did require a change to IB's get_mr function, as well as the protoype. Olaf, if you would like to pull this into your personal tree, I can provide you with delta patches until it is accepted into the OFED 1.4 tree (assuming that you have no major problem with the code below). Signed-Off-By: Jon Mason diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 0751697..0c5028a 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -36,6 +36,7 @@ #include #include #include +#include #include @@ -200,6 +201,43 @@ struct sdp_hah { #define CMA_VERSION 0x00 #define SDP_MAJ_VERSION 0x2 +struct ib_device *ipaddr_to_ibdev(u32 addr) +{ + struct rdma_dev_addr dev_addr; + struct cma_device *cma_dev; + struct net_device *dev; + union ib_gid gid; + + dev = ip_dev_find(addr); + if (!dev) + goto out; + + rdma_copy_addr(&dev_addr, dev, NULL); + + switch (dev->type) { + case ARPHRD_INFINIBAND: + ib_addr_get_sgid(&dev_addr, &gid); + break; + case ARPHRD_ETHER: + iw_addr_get_sgid(&dev_addr, &gid); + break; + default: + goto out; + } + + list_for_each_entry(cma_dev, &dev_list, list) { + int ret; + u8 port; + + ret = ib_find_cached_gid(cma_dev->device, &gid, &port, NULL); + if (!ret) + return cma_dev->device; + } +out: + return NULL; +} +EXPORT_SYMBOL(ipaddr_to_ibdev); + static int cma_comp(struct rdma_id_private *id_priv, enum cma_state comp) { unsigned long flags; diff --git a/net/rds/Kconfig b/net/rds/Kconfig index 57efde6..08edb04 100644 --- a/net/rds/Kconfig +++ b/net/rds/Kconfig @@ -10,6 +10,10 @@ config RDS_IB tristate " RDS over Infiniband" depends RDS +config RDS_IWARP + tristate " RDS over iWARP" + depends RDS + config RDS_TCP tristate " RDS over TCP" depends RDS diff --git a/net/rds/Makefile b/net/rds/Makefile index 6f0d72f..d647eb7 100644 --- a/net/rds/Makefile +++ b/net/rds/Makefile @@ -16,6 +16,12 @@ rds-y += ib.o ib_cm.o ib_recv.o ib_ring.o ib_send.o ib_stats.o \ ib_sysctl.o ib_rdma.o endif +# we don't *quite* have modular transports yet +ifeq ($(CONFIG_RDS_IWARP), m) +rds-y += iwarp.o iwarp_cm.o iwarp_recv.o iwarp_ring.o iwarp_send.o iwarp_stats.o \ + iwarp_sysctl.o iwarp_rdma.o +endif + ifeq ($(CONFIG_RDS_DEBUG), y) EXTRA_CFLAGS += -DDEBUG endif diff --git a/net/rds/ib.c b/net/rds/ib.c index cd2dc7c..1b41ef1 100644 --- a/net/rds/ib.c +++ b/net/rds/ib.c @@ -64,6 +64,9 @@ void rds_ib_add_one(struct ib_device *device) struct rds_ib_device *rds_ibdev; struct ib_device_attr *dev_attr; + if (device->node_type == RDMA_NODE_RNIC) + return; + dev_attr = kmalloc(sizeof *dev_attr, GFP_KERNEL); if (!dev_attr) return; diff --git a/net/rds/ib.h b/net/rds/ib.h index 947977c..f1d3be3 100644 --- a/net/rds/ib.h +++ b/net/rds/ib.h @@ -256,7 +256,7 @@ int ib_update_ipaddr_for_device(struct rds_ib_device *rds_ibdev, __be32 ipaddr); struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *); void rds_ib_destroy_mr_pool(struct rds_ib_mr_pool *); void *rds_ib_get_mr(struct scatterlist *sg, unsigned long nents, - __be32 ip_addr, u32 *key_ret); + struct rds_sock *rs, u32 *key_ret); void rds_ib_sync_mr(void *trans_private, int dir); void rds_ib_free_mr(void *trans_private, int invalidate); void rds_ib_flush_mrs(void); diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index 20c888d..6287fb6 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -609,7 +609,7 @@ int rds_ib_conn_connect(struct rds_connection *conn) dest.sin_family = AF_INET; dest.sin_addr.s_addr = (__force u32)conn->c_faddr; - dest.sin_port = (__force u16)htons(RDS_PORT); + dest.sin_port = (__force u16)htons(RDS_IB_PORT); ret = rdma_resolve_addr(ic->i_cm_id, (struct sockaddr *)&src, (struct sockaddr *)&dest, @@ -794,7 +794,7 @@ int __init rds_ib_listen_init(void) sin.sin_family = PF_INET, sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); - sin.sin_port = (__force u16)htons(RDS_PORT); + sin.sin_port = (__force u16)htons(RDS_IB_PORT); /* * XXX I bet this binds the cm_id to a device. If we want to support @@ -814,7 +814,7 @@ int __init rds_ib_listen_init(void) goto out; } - rdsdebug("cm %p listening on port %u\n", cm_id, RDS_PORT); + rdsdebug("cm %p listening on port %u\n", cm_id, RDS_IB_PORT); rds_ib_listen_id = cm_id; cm_id = NULL; diff --git a/net/rds/ib_rdma.c b/net/rds/ib_rdma.c index 8c6cac3..b427e73 100644 --- a/net/rds/ib_rdma.c +++ b/net/rds/ib_rdma.c @@ -523,10 +523,11 @@ void rds_ib_flush_mrs(void) } void *rds_ib_get_mr(struct scatterlist *sg, unsigned long nents, - __be32 ip_addr, u32 *key_ret) + struct rds_sock *rs, u32 *key_ret) { struct rds_ib_device *rds_ibdev; struct rds_ib_mr *ibmr = NULL; + __be32 ip_addr = rs->rs_bound_addr; int ret; rds_ibdev = ib_get_device(ip_addr); diff --git a/net/rds/iwarp.c b/net/rds/iwarp.c new file mode 100644 index 0000000..8491839 --- /dev/null +++ b/net/rds/iwarp.c @@ -0,0 +1,247 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include +#include +#include +#include +#include +#include + +#include "rds.h" +#include "iwarp.h" + +unsigned int fast_mr_pool_size = RDS_FMR_POOL_SIZE; +unsigned int fast_mr_message_size = RDS_FMR_SIZE + 1; /* +1 allows for unaligned MRs */ + +module_param(fast_mr_pool_size, int, 0444); +MODULE_PARM_DESC(fast_mr_pool_size, " Max number of fmr per HCA"); +module_param(fast_mr_message_size, int, 0444); +MODULE_PARM_DESC(fast_mr_message_size, " Max size of a RDMA transfer"); + +struct list_head rds_iwarp_devices; + +void rds_iwarp_add_one(struct ib_device *device); +void rds_iwarp_remove_one(struct ib_device *device); + +struct ib_client rds_iwarp_client = { + .name = "rds_iwarp", + .add = rds_iwarp_add_one, + .remove = rds_iwarp_remove_one +}; + +void rds_iwarp_add_one(struct ib_device *device) +{ + struct rds_iwarp_device *rds_iwarpdev; + struct ib_device_attr *dev_attr; + + if (device->node_type != RDMA_NODE_RNIC) + return; + + dev_attr = kmalloc(sizeof *dev_attr, GFP_KERNEL); + if (!dev_attr) + return; + + if (ib_query_device(device, dev_attr)) { + rdsdebug("Query device failed for %s\n", device->name); + goto free_attr; + } + + rds_iwarpdev = kmalloc(sizeof *rds_iwarpdev, GFP_KERNEL); + if (!rds_iwarpdev) + goto free_attr; + + spin_lock_init(&rds_iwarpdev->spinlock); + + rds_iwarpdev->max_sge = min(dev_attr->max_sge, RDS_IWARP_MAX_SGE); + + rds_iwarpdev->fmr_page_shift = max(9, ffs(dev_attr->page_size_cap) - 1); + rds_iwarpdev->fmr_page_size = 1 << rds_iwarpdev->fmr_page_shift; + rds_iwarpdev->fmr_page_mask = ~((u64) rds_iwarpdev->fmr_page_size - 1); + rds_iwarpdev->fmr_max_remaps = dev_attr->max_map_per_fmr?: 32; + rds_iwarpdev->max_fmrs = dev_attr->max_fmr? + min_t(unsigned int, dev_attr->max_fmr, fast_mr_pool_size) : + fast_mr_pool_size; + + rds_iwarpdev->dev = device; + rds_iwarpdev->pd = ib_alloc_pd(device); + if (IS_ERR(rds_iwarpdev->pd)) + goto free_dev; + + rds_iwarpdev->mr = ib_get_dma_mr(rds_iwarpdev->pd, IB_ACCESS_REMOTE_READ | + IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE); + if (IS_ERR(rds_iwarpdev->mr)) + goto err_pd; + + rds_iwarpdev->mr_pool = rds_iwarp_create_mr_pool(rds_iwarpdev); + if (IS_ERR(rds_iwarpdev->mr_pool)) { + rds_iwarpdev->mr_pool = NULL; + goto err_mr; + } + + INIT_LIST_HEAD(&rds_iwarpdev->ipaddr_list); + list_add_tail(&rds_iwarpdev->list, &rds_iwarp_devices); + + ib_set_client_data(device, &rds_iwarp_client, rds_iwarpdev); + + goto free_attr; + +err_mr: + ib_dereg_mr(rds_iwarpdev->mr); +err_pd: + ib_dealloc_pd(rds_iwarpdev->pd); +free_dev: + kfree(rds_iwarpdev); +free_attr: + kfree(dev_attr); +} + +void rds_iwarp_remove_one(struct ib_device *device) +{ + struct rds_iwarp_device *rds_iwarpdev; + struct rds_iwarp_ipaddr *i_ipaddr, *next; + + rds_iwarpdev = ib_get_client_data(device, &rds_iwarp_client); + if (!rds_iwarpdev) + return; + + list_for_each_entry_safe(i_ipaddr, next, &rds_iwarpdev->ipaddr_list, list) { + list_del(&i_ipaddr->list); + kfree(i_ipaddr); + } + + if (rds_iwarpdev->mr_pool) + rds_iwarp_destroy_mr_pool(rds_iwarpdev->mr_pool); + + ib_dereg_mr(rds_iwarpdev->mr); + ib_dealloc_pd(rds_iwarpdev->pd); + + list_del(&rds_iwarpdev->list); + kfree(rds_iwarpdev); +} + +extern struct ib_device *ipaddr_to_ibdev(u32 addr); + +static int rds_iwarp_laddr_check(__be32 addr) +{ + struct ib_device *ibdev; + int ret = -EADDRNOTAVAIL; + + ibdev = ipaddr_to_ibdev(addr); + if (!ibdev) + goto out; + + if (ibdev->node_type == RDMA_NODE_RNIC) { + ret = 0; + } + +out: + rdsdebug("addr %u.%u.%u.%u ret %d\n", NIPQUAD(addr), ret); + return ret; +} + +/* + * conns should have been freed up by the time we get here.. + */ +static void rds_iwarp_exit(void) +{ + rds_iwarp_listen_stop(); + rds_trans_unregister(&rds_iwarp_transport); + rds_iwarp_recv_exit(); + rds_iwarp_sysctl_exit(); + ib_unregister_client(&rds_iwarp_client); +} + +struct rds_transport rds_iwarp_transport = { + .laddr_check = rds_iwarp_laddr_check, + .xmit_complete = rds_iwarp_xmit_complete, + .xmit = rds_iwarp_xmit, + .xmit_cong_map = NULL, + .xmit_rdma = rds_iwarp_xmit_rdma, + .recv = rds_iwarp_recv, + .conn_alloc = rds_iwarp_conn_alloc, + .conn_free = rds_iwarp_conn_free, + .conn_connect = rds_iwarp_conn_connect, + .conn_shutdown = rds_iwarp_conn_shutdown, + .inc_copy_to_user = rds_iwarp_inc_copy_to_user, + .inc_purge = rds_iwarp_inc_purge, + .inc_free = rds_iwarp_inc_free, + .listen_stop = rds_iwarp_listen_stop, + .stats_info_copy = rds_iwarp_stats_info_copy, + .exit = rds_iwarp_exit, + .get_mr = rds_iwarp_get_mr, + .sync_mr = rds_iwarp_sync_mr, + .free_mr = rds_iwarp_free_mr, + .flush_mrs = rds_iwarp_flush_mrs, + .t_owner = THIS_MODULE, + .t_name = "iwarp", +}; + +int __init rds_iwarp_init(void) +{ + int ret; + + INIT_LIST_HEAD(&rds_iwarp_devices); + + ret = ib_register_client(&rds_iwarp_client); + if (ret) + goto out; + + ret = rds_iwarp_sysctl_init(); + if (ret) + goto out_iwarpreg; + + ret = rds_iwarp_recv_init(); + if (ret) + goto out_sysctl; + + ret = rds_trans_register(&rds_iwarp_transport); + if (ret) + goto out_recv; + + ret = rds_iwarp_listen_init(); + if (ret) + goto out_register; + + goto out; + +out_register: + rds_trans_unregister(&rds_iwarp_transport); +out_recv: + rds_iwarp_recv_exit(); +out_sysctl: + rds_iwarp_sysctl_exit(); +out_iwarpreg: + ib_unregister_client(&rds_iwarp_client); +out: + return ret; +} diff --git a/net/rds/iwarp.h b/net/rds/iwarp.h new file mode 100644 index 0000000..cc07f2d --- /dev/null +++ b/net/rds/iwarp.h @@ -0,0 +1,339 @@ +#ifndef _RDS_IWARP_H +#define _RDS_IWARP_H + +#include +#include +#include "rds.h" + +#define RDS_IWARP_RESOLVE_TIMEOUT_MS 5000 + +/* FIXME - use ib_query_device to determine proper value */ +#define RDS_FMR_SIZE 20 +#define RDS_FMR_POOL_SIZE 2048 + +#define RDS_IWARP_MAX_SGE 8 +#define RDS_IWARP_RECV_SGE 2 + +/* FIXME - call ib_query_device to determine sane values for this based on the HW */ +#define RDS_IWARP_DEFAULT_RECV_WR 512 +#define RDS_IWARP_DEFAULT_SEND_WR 128 + +#define RDS_IWARP_SUPPORTED_PROTOCOLS 0x00000003 /* minor versions supported */ + +/* + * IWARP posts RDS_FRAG_SIZE fragments of pages to the receive queues to + * try and minimize the amount of memory tied up both the device and + * socket receive queues. + */ +/* page offset of the final full frag that fits in the page */ +#define RDS_PAGE_LAST_OFF (((PAGE_SIZE / RDS_FRAG_SIZE) - 1) * RDS_FRAG_SIZE) +struct rds_page_frag { + struct list_head f_item; + struct page *f_page; + unsigned long f_offset; + dma_addr_t f_mapped; +}; + +struct rds_iwarp_incoming { + struct list_head ii_frags; + struct rds_incoming ii_inc; +}; + +struct rds_iwarp_connect_private { + /* Add new fields at the end, and don't permute existing fields. */ + __be32 dp_saddr; + __be32 dp_daddr; + u8 dp_protocol_major; + u8 dp_protocol_minor; + __be16 dp_protocol_minor_mask; /* bitmask */ + __be32 dp_reserved1; + __be64 dp_ack_seq; + __be32 dp_credit; /* non-zero enables flow ctl */ +}; + +struct rds_iwarp_send_work { + struct rds_message *s_rm; + struct rds_rdma_op *s_op; + struct ib_send_wr s_wr; + struct ib_sge s_sge[RDS_IWARP_MAX_SGE]; + unsigned long s_queued; +}; + +struct rds_iwarp_recv_work { + struct rds_iwarp_incoming *r_iwarpinc; + struct rds_page_frag *r_frag; + struct ib_recv_wr r_wr; + struct ib_sge r_sge[2]; +}; + +struct rds_iwarp_work_ring { + u32 w_nr; + u32 w_alloc_ptr; + u32 w_alloc_ctr; + u32 w_free_ptr; + atomic_t w_free_ctr; +}; + +struct rds_iwarp_connection { + /* alphabet soup, IWARPTA style */ + struct rdma_cm_id *i_cm_id; + struct ib_pd *i_pd; + struct ib_mr *i_mr; + struct ib_cq *i_send_cq; + struct ib_cq *i_recv_cq; + + /* tx */ + struct rds_iwarp_work_ring i_send_ring; + struct rds_message *i_rm; + struct rds_header *i_send_hdrs; + u64 i_send_hdrs_dma; + struct rds_iwarp_send_work *i_sends; + + /* rx */ + struct mutex i_recv_mutex; + struct rds_iwarp_work_ring i_recv_ring; + struct rds_iwarp_incoming *i_iwarpinc; + u32 i_recv_data_rem; + struct rds_header *i_recv_hdrs; + u64 i_recv_hdrs_dma; + struct rds_iwarp_recv_work *i_recvs; + struct rds_page_frag i_frag; + u64 i_ack_recv; /* last ACK received */ + + /* sending acks */ + unsigned long i_ack_flags; +#ifndef KERNEL_HAS_ATOMIC64 + spinlock_t i_ack_lock; + u64 i_ack_next; /* next ACK to send */ +#else + atomic64_t i_ack_next; /* next ACK to send */ +#endif + struct rds_header *i_ack; + struct ib_send_wr i_ack_wr; + struct ib_sge i_ack_sge; + u64 i_ack_dma; + unsigned long i_ack_queued; + + /* Flow control related information + * + * Our algorithm uses a pair variables that we need to access + * atomically - one for the send credits, and one posted + * recv credits we need to transfer to remote. + * Rather than protect them using a slow spinlock, we put both into + * a single atomic_t and update it using cmpxchg + */ + atomic_t i_credits; + + /* Protocol version specific information */ + unsigned int i_hdr_idx; /* 1 (old) or 0 (3.1 or later) */ + unsigned int i_flowctl : 1; /* enable/disable flow ctl */ + + /* Batched completions */ + unsigned int i_unsignaled_wrs; + long i_unsignaled_bytes; +}; + +/* This assumes that atomic_t is at least 32 bits */ +#define IWARP_GET_SEND_CREDITS(v) ((v) & 0xffff) +#define IWARP_GET_POST_CREDITS(v) ((v) >> 16) +#define IWARP_SET_SEND_CREDITS(v) ((v) & 0xffff) +#define IWARP_SET_POST_CREDITS(v) ((v) << 16) + +struct rds_iwarp_ipaddr { + struct list_head list; + __be32 ipaddr; +}; + +struct rds_iwarp_device { + struct list_head list; + struct list_head ipaddr_list; + struct ib_device *dev; + struct ib_pd *pd; + struct ib_mr *mr; + struct rds_iwarp_mr_pool *mr_pool; + int fmr_page_shift; + int fmr_page_size; + u64 fmr_page_mask; + unsigned int fmr_max_remaps; + unsigned int max_fmrs; + int max_sge; + spinlock_t spinlock; +}; + +/* bits for i_ack_flags */ +#define IWARP_ACK_IN_FLIGHT 0 +#define IWARP_ACK_REQUESTED 1 + +/* Magic WR_ID for ACKs */ +#define RDS_IWARP_ACK_WR_ID (~(u64) 0) + +struct rds_iwarp_statistics { + unsigned long s_iwarp_connect_raced; + unsigned long s_iwarp_listen_closed_stale; + unsigned long s_iwarp_tx_cq_call; + unsigned long s_iwarp_tx_cq_event; + unsigned long s_iwarp_tx_ring_full; + unsigned long s_iwarp_tx_throttle; + unsigned long s_iwarp_tx_sg_mapping_failure; + unsigned long s_iwarp_tx_stalled; + unsigned long s_iwarp_tx_credit_updates; + unsigned long s_iwarp_rx_cq_call; + unsigned long s_iwarp_rx_cq_event; + unsigned long s_iwarp_rx_ring_empty; + unsigned long s_iwarp_rx_refill_from_cq; + unsigned long s_iwarp_rx_refill_from_thread; + unsigned long s_iwarp_rx_alloc_limit; + unsigned long s_iwarp_rx_credit_updates; + unsigned long s_iwarp_ack_sent; + unsigned long s_iwarp_ack_send_failure; + unsigned long s_iwarp_ack_send_delayed; + unsigned long s_iwarp_ack_send_piggybacked; + unsigned long s_iwarp_ack_received; + unsigned long s_iwarp_rdma_mr_alloc; + unsigned long s_iwarp_rdma_mr_free; + unsigned long s_iwarp_rdma_mr_used; + unsigned long s_iwarp_rdma_mr_pool_flush; + unsigned long s_iwarp_rdma_mr_pool_wait; +}; + +extern struct workqueue_struct *rds_iwarp_wq; + +/* + * Fake ib_dma_sync_sg_for_{cpu,device} as long as ib_verbs.h + * doesn't define it. + */ +static inline void rds_iwarp_dma_sync_sg_for_cpu(struct ib_device *dev, + struct scatterlist *sg, unsigned int sg_dma_len, int direction) +{ + unsigned int i; + + for (i = 0; i < sg_dma_len; ++i) { + ib_dma_sync_single_for_cpu(dev, + ib_sg_dma_address(dev, &sg[i]), + ib_sg_dma_len(dev, &sg[i]), + direction); + } +} +#define ib_dma_sync_sg_for_cpu rds_iwarp_dma_sync_sg_for_cpu + +static void inline rds_iwarp_dma_sync_sg_for_device(struct ib_device *dev, + struct scatterlist *sg, unsigned int sg_dma_len, int direction) +{ + unsigned int i; + + for (i = 0; i < sg_dma_len; ++i) { + ib_dma_sync_single_for_device(dev, + ib_sg_dma_address(dev, &sg[i]), + ib_sg_dma_len(dev, &sg[i]), + direction); + } +} +#define ib_dma_sync_sg_for_device rds_iwarp_dma_sync_sg_for_device + + +/* iwarp.c */ +extern struct rds_transport rds_iwarp_transport; +extern void rds_iwarp_add_one(struct ib_device *device); +extern void rds_iwarp_remove_one(struct ib_device *device); +extern struct ib_client rds_iwarp_client; + +extern unsigned int fast_mr_pool_size; +extern unsigned int fast_mr_message_size; + +/* iwarp_cm.c */ +int rds_iwarp_conn_alloc(struct rds_connection *conn, gfp_t gfp); +void rds_iwarp_conn_free(void *arg); +int rds_iwarp_conn_connect(struct rds_connection *conn); +void rds_iwarp_conn_shutdown(struct rds_connection *conn); +void rds_iwarp_state_change(struct sock *sk); +int __init rds_iwarp_listen_init(void); +void rds_iwarp_listen_stop(void); +void __rds_iwarp_conn_error(struct rds_connection *conn, const char *, ...); + +#define rds_iwarp_conn_error(conn, fmt...) \ + __rds_iwarp_conn_error(conn, KERN_WARNING "RDS/IWARP: " fmt ) + +/* iwarp_rdma.c */ +struct rds_iwarp_mr_pool *rds_iwarp_create_mr_pool(struct rds_iwarp_device *); +void rds_iwarp_destroy_mr_pool(struct rds_iwarp_mr_pool *); +void *rds_iwarp_get_mr(struct scatterlist *sg, unsigned long nents, + struct rds_sock *rs, u32 *key_ret); +void rds_iwarp_sync_mr(void *trans_private, int dir); +void rds_iwarp_free_mr(void *trans_private, int invalidate); +void rds_iwarp_flush_mrs(void); + +/* iwarp_recv.c */ +int __init rds_iwarp_recv_init(void); +void rds_iwarp_recv_exit(void); +int rds_iwarp_recv(struct rds_connection *conn); +int rds_iwarp_recv_refill(struct rds_connection *conn, gfp_t kptr_gfp, + gfp_t page_gfp, int prefill); +void rds_iwarp_inc_purge(struct rds_incoming *inc); +void rds_iwarp_inc_free(struct rds_incoming *inc); +int rds_iwarp_inc_copy_to_user(struct rds_incoming *inc, struct iovec *iov, + size_t size); +void rds_iwarp_recv_cq_comp_handler(struct ib_cq *cq, void *context); +void rds_iwarp_recv_init_ring(struct rds_iwarp_connection *ic); +void rds_iwarp_recv_clear_ring(struct rds_iwarp_connection *ic); +void rds_iwarp_recv_init_ack(struct rds_iwarp_connection *ic); +void rds_iwarp_attempt_ack(struct rds_iwarp_connection *ic); +void rds_iwarp_ack_send_complete(struct rds_iwarp_connection *ic); +u64 rds_iwarp_piggyb_ack(struct rds_iwarp_connection *ic); + +/* iwarp_ring.c */ +void rds_iwarp_ring_init(struct rds_iwarp_work_ring *ring, u32 nr); +u32 rds_iwarp_ring_alloc(struct rds_iwarp_work_ring *ring, u32 val, u32 *pos); +void rds_iwarp_ring_free(struct rds_iwarp_work_ring *ring, u32 val); +void rds_iwarp_ring_unalloc(struct rds_iwarp_work_ring *ring, u32 val); +int rds_iwarp_ring_empty(struct rds_iwarp_work_ring *ring); +u32 rds_iwarp_ring_oldest(struct rds_iwarp_work_ring *ring); +u32 rds_iwarp_ring_completed(struct rds_iwarp_work_ring *ring, u32 wr_id, u32 oldest); +extern wait_queue_head_t rds_iwarp_ring_empty_wait; + +/* iwarp_send.c */ +void rds_iwarp_xmit_complete(struct rds_connection *conn); +int rds_iwarp_xmit(struct rds_connection *conn, struct rds_message *rm, + unsigned int hdr_off, unsigned int sg, unsigned int off); +void rds_iwarp_send_cq_comp_handler(struct ib_cq *cq, void *context); +void rds_iwarp_send_init_ring(struct rds_iwarp_connection *ic); +void rds_iwarp_send_clear_ring(struct rds_iwarp_connection *ic); +int rds_iwarp_xmit_rdma(struct rds_connection *conn, struct rds_rdma_op *op); +void rds_iwarp_send_add_credits(struct rds_connection *conn, unsigned int credits); +void rds_iwarp_advertise_credits(struct rds_connection *conn, unsigned int posted); +int rds_iwarp_send_grab_credits(struct rds_iwarp_connection *ic, u32 wanted, + u32 *adv_credits); + +/* iwarp_stats.c */ +RDS_DECLARE_PER_CPU(struct rds_iwarp_statistics, rds_iwarp_stats); +#define rds_iwarp_stats_inc(member) rds_stats_inc_which(rds_iwarp_stats, member) +unsigned int rds_iwarp_stats_info_copy(struct rds_info_iterator *iter, + unsigned int avail); + +/* iwarp_sysctl.c */ +int __init rds_iwarp_sysctl_init(void); +void rds_iwarp_sysctl_exit(void); +extern unsigned long rds_iwarp_sysctl_max_send_wr; +extern unsigned long rds_iwarp_sysctl_max_recv_wr; +extern unsigned long rds_iwarp_sysctl_max_unsig_wrs; +extern unsigned long rds_iwarp_sysctl_max_unsig_bytes; +extern unsigned long rds_iwarp_sysctl_max_recv_allocation; +extern unsigned int rds_iwarp_sysctl_flow_control; +extern ctl_table rds_iwarp_sysctl_table[]; + +/* + * Helper functions for getting/setting the header and data SGEs in + * RDS packets (not RDMA) + */ +static inline struct ib_sge * +rds_iwarp_header_sge(struct rds_iwarp_connection *ic, struct ib_sge *sge) +{ + return &sge[ic->i_hdr_idx]; +} + +static inline struct ib_sge * +rds_iwarp_data_sge(struct rds_iwarp_connection *ic, struct ib_sge *sge) +{ + return &sge[1 - ic->i_hdr_idx]; +} + +#endif diff --git a/net/rds/iwarp_cm.c b/net/rds/iwarp_cm.c new file mode 100644 index 0000000..ecd08dc --- /dev/null +++ b/net/rds/iwarp_cm.c @@ -0,0 +1,853 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include +#include + +#include "rds.h" +#include "iwarp.h" + +static struct rdma_cm_id *rds_iwarp_listen_id; + +/* + * Set the selected protocol version + */ +static void rds_iwarp_set_protocol(struct rds_connection *conn, unsigned int version) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + + conn->c_version = version; + + if (conn->c_version == RDS_PROTOCOL_3_0) { + ic->i_hdr_idx = 1; + } else { + ic->i_hdr_idx = 0; + } +} + +/* + * Set up flow control + */ +static void rds_iwarp_set_flow_control(struct rds_connection *conn, u32 credits) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + + if (rds_iwarp_sysctl_flow_control && credits != 0) { + /* We're doing flow control */ + ic->i_flowctl = 1; + rds_iwarp_send_add_credits(conn, credits); + } else { + ic->i_flowctl = 0; + } +} + +static int iwarp_update_ipaddr_for_device(struct rds_iwarp_device *rds_iwarpdev, __be32 ipaddr) +{ + struct rds_iwarp_ipaddr *i_ipaddr; + + spin_lock_irq(&rds_iwarpdev->spinlock); + list_for_each_entry(i_ipaddr, &rds_iwarpdev->ipaddr_list, list) { + if (i_ipaddr->ipaddr == ipaddr) { + spin_unlock_irq(&rds_iwarpdev->spinlock); + return 0; + } + } + spin_unlock_irq(&rds_iwarpdev->spinlock); + + i_ipaddr = kmalloc(sizeof *i_ipaddr, GFP_KERNEL); + if (!i_ipaddr) + return -ENOMEM; + + i_ipaddr->ipaddr = ipaddr; + + spin_lock_irq(&rds_iwarpdev->spinlock); + list_add_tail(&i_ipaddr->list, &rds_iwarpdev->ipaddr_list); + spin_unlock_irq(&rds_iwarpdev->spinlock); + + return 0; +} + +/* + * Connection established. + * We get here for both outgoing and incoming connection. + */ +static void rds_iwarp_connect_complete(struct rds_connection *conn, struct rdma_cm_event *event) +{ + const struct rds_iwarp_connect_private *dp = NULL; + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct rds_iwarp_device *rds_iwarpdev; + struct ib_qp_attr qp_attr; + int ret; + + if (event->param.conn.private_data_len) { + dp = event->param.conn.private_data; + + rds_iwarp_set_protocol(conn, + RDS_PROTOCOL(dp->dp_protocol_major, + dp->dp_protocol_minor)); + rds_iwarp_set_flow_control(conn, be32_to_cpu(dp->dp_credit)); + } + + printk(KERN_NOTICE "RDS/IWARP: connected to %u.%u.%u.%u version %u.%u%s\n", + NIPQUAD(conn->c_laddr), + RDS_PROTOCOL_MAJOR(conn->c_version), + RDS_PROTOCOL_MINOR(conn->c_version), + ic->i_flowctl? ", flow control" : ""); + + rds_iwarp_recv_refill(conn, GFP_KERNEL, GFP_HIGHUSER, 1); + + /* Tune the RNR timeout. We use a rather low timeout, but + * not the absolute minimum - this should be tunable. + * + * We already set the RNR retry count to 7 (which is the + * smallest infinite number :-) above + */ + qp_attr.qp_state = IB_QPS_RTS; + qp_attr.min_rnr_timer = IB_RNR_TIMER_000_32; + ret = ib_modify_qp(ic->i_cm_id->qp, &qp_attr, + IB_QP_STATE | IB_QP_MIN_RNR_TIMER); + if (ret) { + printk(KERN_NOTICE "ib_modify_qp(IB_QP_MIN_RNR_TIMER, %u): err=%d\n", + qp_attr.min_rnr_timer, -ret); + } + + /* update ib_device with this local ipaddr */ + rds_iwarpdev = ib_get_client_data(ic->i_cm_id->device, &rds_iwarp_client); + iwarp_update_ipaddr_for_device(rds_iwarpdev, conn->c_laddr); + + /* If the peer gave us the last packet it saw, process this as if + * we had received a regular ACK. */ + if (dp && dp->dp_ack_seq) + rds_send_drop_acked(conn, be64_to_cpu(dp->dp_ack_seq), NULL); + + rds_connect_complete(conn); +} + +static void rds_iwarp_cm_fill_conn_param(struct rds_connection *conn, + struct rdma_conn_param *conn_param, + struct rds_iwarp_connect_private *dp, + u32 protocol_version) +{ + memset(conn_param, 0, sizeof(struct rdma_conn_param)); + /* XXX tune these? */ + conn_param->responder_resources = 1; + conn_param->initiator_depth = 1; + + if (dp) { + struct rds_iwarp_connection *ic = conn->c_transport_data; + + memset(dp, 0, sizeof(*dp)); + dp->dp_saddr = conn->c_laddr; + dp->dp_daddr = conn->c_faddr; + dp->dp_protocol_major = RDS_PROTOCOL_MAJOR(protocol_version); + dp->dp_protocol_minor = RDS_PROTOCOL_MINOR(protocol_version); + dp->dp_protocol_minor_mask = cpu_to_be16(RDS_IWARP_SUPPORTED_PROTOCOLS); + dp->dp_ack_seq = rds_iwarp_piggyb_ack(ic); + + /* Advertise flow control. + * + * Major chicken and egg alert! + * We would like to post receive buffers before we get here (eg. + * in rds_iwarp_setup_qp), so that we can give the peer an accurate + * credit value. + * Unfortunately we can't post receive buffers until we've finished + * protocol negotiation, and know in which order data and payload + * are arranged. + * + * What we do here is we give the peer a small initial credit, and + * initialize the number of posted buffers to a negative value. + */ + if (ic->i_flowctl) { + atomic_set(&ic->i_credits, IWARP_SET_POST_CREDITS(-4)); + dp->dp_credit = cpu_to_be32(4); + } + + conn_param->private_data = dp; + conn_param->private_data_len = sizeof(*dp); + } +} + +static void rds_iwarp_cq_event_handler(struct ib_event *event, void *data) +{ + rdsdebug("event %u data %p\n", event->event, data); +} + +static void rds_iwarp_qp_event_handler(struct ib_event *event, void *data) +{ + struct rds_connection *conn = data; + struct rds_iwarp_connection *ic = conn->c_transport_data; + + rdsdebug("conn %p ic %p event %u\n", conn, ic, event->event); + + switch (event->event) { + case IB_EVENT_COMM_EST: + rdma_notify(ic->i_cm_id, IB_EVENT_COMM_EST); + break; + default: + printk(KERN_WARNING "RDS/IWARP: unhandled QP event %u " + "on connection to %u.%u.%u.%u\n", event->event, + NIPQUAD(conn->c_faddr)); + break; + } +} + +/* + * This needs to be very careful to not leave IS_ERR pointers around for + * cleanup to trip over. + */ +static int rds_iwarp_setup_qp(struct rds_connection *conn) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct ib_device *dev = ic->i_cm_id->device; + struct ib_qp_init_attr attr; + struct rds_iwarp_device *rds_iwarpdev; + int ret; + + /* rds_iwarp_add_one creates a rds_iwarp_device object per IB device, + * and allocates a protection domain, memory range and FMR pool + * for each. If that fails for any reason, it will not register + * the rds_iwarpdev at all. + */ + rds_iwarpdev = ib_get_client_data(dev, &rds_iwarp_client); + if (rds_iwarpdev == NULL) { + if (printk_ratelimit()) + printk(KERN_NOTICE "RDS/IWARP: No client_data for device %s\n", + dev->name); + return -EOPNOTSUPP; + } + + /* Protection domain and memory range */ + ic->i_pd = rds_iwarpdev->pd; + ic->i_mr = rds_iwarpdev->mr; + + ic->i_send_cq = ib_create_cq(dev, rds_iwarp_send_cq_comp_handler, + rds_iwarp_cq_event_handler, conn, + ic->i_send_ring.w_nr + 1, 0); + if (IS_ERR(ic->i_send_cq)) { + ret = PTR_ERR(ic->i_send_cq); + ic->i_send_cq = NULL; + rdsdebug("ib_create_cq send failed: %d\n", ret); + goto out; + } + + ic->i_recv_cq = ib_create_cq(dev, rds_iwarp_recv_cq_comp_handler, + rds_iwarp_cq_event_handler, conn, + ic->i_recv_ring.w_nr, 0); + if (IS_ERR(ic->i_recv_cq)) { + ret = PTR_ERR(ic->i_recv_cq); + ic->i_recv_cq = NULL; + rdsdebug("ib_create_cq recv failed: %d\n", ret); + goto out; + } + + ret = ib_req_notify_cq(ic->i_send_cq, IB_CQ_NEXT_COMP); + if (ret) { + rdsdebug("ib_req_notify_cq send failed: %d\n", ret); + goto out; + } + + ret = ib_req_notify_cq(ic->i_recv_cq, IB_CQ_SOLICITED); + if (ret) { + rdsdebug("ib_req_notify_cq recv failed: %d\n", ret); + goto out; + } + + /* XXX negotiate max send/recv with remote? */ + memset(&attr, 0, sizeof(attr)); + attr.event_handler = rds_iwarp_qp_event_handler; + attr.qp_context = conn; + /* + 1 to allow for the single ack message */ + attr.cap.max_send_wr = ic->i_send_ring.w_nr + 1; + attr.cap.max_recv_wr = ic->i_recv_ring.w_nr + 1; + attr.cap.max_send_sge = rds_iwarpdev->max_sge; + attr.cap.max_recv_sge = RDS_IWARP_RECV_SGE; + attr.sq_sig_type = IB_SIGNAL_REQ_WR; + attr.qp_type = IB_QPT_RC; + attr.send_cq = ic->i_send_cq; + attr.recv_cq = ic->i_recv_cq; + + /* + * XXX this can fail if max_*_wr is too large? Are we supposed + * to back off until we get a value that the hardware can support? + */ + ret = rdma_create_qp(ic->i_cm_id, ic->i_pd, &attr); + if (ret) { + rdsdebug("rdma_create_qp failed: %d\n", ret); + goto out; + } + + ic->i_send_hdrs = ib_dma_alloc_coherent(dev, + ic->i_send_ring.w_nr * + sizeof(struct rds_header), + &ic->i_send_hdrs_dma, GFP_KERNEL); + if (ic->i_send_hdrs == NULL) { + ret = -ENOMEM; + rdsdebug("ib_dma_alloc_coherent send failed\n"); + goto out; + } + + ic->i_recv_hdrs = ib_dma_alloc_coherent(dev, + ic->i_recv_ring.w_nr * + sizeof(struct rds_header), + &ic->i_recv_hdrs_dma, GFP_KERNEL); + if (ic->i_recv_hdrs == NULL) { + ret = -ENOMEM; + rdsdebug("ib_dma_alloc_coherent recv failed\n"); + goto out; + } + + ic->i_ack = ib_dma_alloc_coherent(dev, sizeof(struct rds_header), + &ic->i_ack_dma, GFP_KERNEL); + if (ic->i_ack == NULL) { + ret = -ENOMEM; + rdsdebug("ib_dma_alloc_coherent ack failed\n"); + goto out; + } + + ic->i_sends = kmalloc(ic->i_send_ring.w_nr * + sizeof(struct rds_iwarp_send_work), GFP_KERNEL); + if (ic->i_sends == NULL) { + ret = -ENOMEM; + rdsdebug("send allocation failed\n"); + goto out; + } + rds_iwarp_send_init_ring(ic); + + ic->i_recvs = kmalloc(ic->i_recv_ring.w_nr * + sizeof(struct rds_iwarp_recv_work), GFP_KERNEL); + if (ic->i_recvs == NULL) { + ret = -ENOMEM; + rdsdebug("recv allocation failed\n"); + goto out; + } + + rds_iwarp_recv_init_ring(ic); + rds_iwarp_recv_init_ack(ic); + + rdsdebug("conn %p pd %p mr %p cq %p %p\n", conn, ic->i_pd, ic->i_mr, + ic->i_send_cq, ic->i_recv_cq); + +out: + return ret; +} + +static u32 rds_iwarp_protocol_compatible(const struct rds_iwarp_connect_private *dp) +{ + u16 common; + u32 version = 0; + + /* rdma_cm private data is odd - when there is any private data in the + * request, we will be given a pretty large buffer without telling us the + * original size. The only way to tell the difference is by looking at + * the contents, which are initialized to zero. + * If the protocol version fields aren't set, this is a connection attempt + * from an older version. This could could be 3.0 or 2.0 - we can't tell. + * We really should have changed this for OFED 1.3 :-( */ + if (dp->dp_protocol_major == 0) + return RDS_PROTOCOL_3_0; + + common = be16_to_cpu(dp->dp_protocol_minor_mask) & RDS_IWARP_SUPPORTED_PROTOCOLS; + if (dp->dp_protocol_major == 3 && common) { + version = RDS_PROTOCOL_3_0; + while ((common >>= 1) != 0) + version++; + } else if (printk_ratelimit()) { + printk(KERN_NOTICE "RDS: Connection from %u.%u.%u.%u using " + "incompatible protocol version %u.%u\n", + NIPQUAD(dp->dp_saddr), + dp->dp_protocol_major, + dp->dp_protocol_minor); + } + return version; +} + +static int rds_iwarp_cm_handle_connect(struct rdma_cm_id *cm_id, + struct rdma_cm_event *event) +{ + const struct rds_iwarp_connect_private *dp = event->param.conn.private_data; + struct rds_iwarp_connect_private dp_rep; + struct rds_connection *conn = NULL; + struct rds_iwarp_connection *ic = NULL; + struct rdma_conn_param conn_param; + struct rds_iwarp_device *rds_iwarpdev; + u32 version; + int err, destroy = 1; + + /* Check whether the remote protocol version matches ours. */ + version = rds_iwarp_protocol_compatible(dp); + if (!version) + goto out; + + rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u\n", + NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), + RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version)); + + conn = rds_conn_create(dp->dp_daddr, dp->dp_saddr, &rds_iwarp_transport, + GFP_KERNEL); + if (IS_ERR(conn)) { + rdsdebug("rds_conn_create failed (%ld)\n", PTR_ERR(conn)); + conn = NULL; + goto out; + } + ic = conn->c_transport_data; + + rds_iwarp_set_protocol(conn, version); + rds_iwarp_set_flow_control(conn, be32_to_cpu(dp->dp_credit)); + + /* If the peer gave us the last packet it saw, process this as if + * we had received a regular ACK. */ + if (dp->dp_ack_seq) + rds_send_drop_acked(conn, be64_to_cpu(dp->dp_ack_seq), NULL); + + /* + * The connection request may occur while the + * previous connection exist, e.g. in case of failover. + * But as connections may be initiated simultaneously + * by both hosts, we have a random backoff mechanism - + * see the comment above rds_queuereconnect() + */ + if (!rds_conn_transition(conn, RDS_CONN_DOWN, RDS_CONN_CONNECTING)) { + if (rds_conn_state(conn) == RDS_CONN_UP) { + rds_iwarp_conn_error(conn, "incoming connect while connecting\n"); + rds_iwarp_stats_inc(s_iwarp_listen_closed_stale); + } else + if (rds_conn_state(conn) == RDS_CONN_CONNECTING) { + /* Wait and see - our connect may still be succeeding */ + rds_iwarp_stats_inc(s_iwarp_connect_raced); + } + goto out; + } + + BUG_ON(cm_id->context); + BUG_ON(ic->i_cm_id); + + ic->i_cm_id = cm_id; + cm_id->context = conn; + + /* We got halfway through setting up the ib_connection, if we + * fail now, we have to take the long route out of this mess. */ + destroy = 0; + + err = rds_iwarp_setup_qp(conn); + if (err) { + rds_iwarp_conn_error(conn, "rds_iwarp_setup_qp failed (%d)\n", err); + goto out; + } + + rds_iwarp_cm_fill_conn_param(conn, &conn_param, &dp_rep, version); + + /* rdma_accept() calls rdma_reject() internally if it fails */ + err = rdma_accept(cm_id, &conn_param); + if (err) { + rds_iwarp_conn_error(conn, "rdma_accept failed (%d)\n", err); + goto out; + } + + /* update ib_device with this local ipaddr */ + rds_iwarpdev = ib_get_client_data(ic->i_cm_id->device, &rds_iwarp_client); + iwarp_update_ipaddr_for_device(rds_iwarpdev, dp->dp_saddr); + + return 0; + +out: + rdma_reject(cm_id, NULL, 0); + return destroy; +} + + +static int rds_iwarp_cm_initiate_connect(struct rdma_cm_id *cm_id) +{ + struct rds_connection *conn = cm_id->context; + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct rdma_conn_param conn_param; + struct rds_iwarp_connect_private dp; + int ret; + + /* If the peer doesn't do protocol negotiation, we must + * default to RDSv3.0 */ + rds_iwarp_set_protocol(conn, RDS_PROTOCOL_3_0); + ic->i_flowctl = rds_iwarp_sysctl_flow_control; /* advertise flow control */ + + ret = rds_iwarp_setup_qp(conn); + if (ret) { + rds_iwarp_conn_error(conn, "1rds_iwarp_setup_qp failed (%d)\n", ret); + goto out; + } + + rds_iwarp_cm_fill_conn_param(conn, &conn_param, &dp, RDS_PROTOCOL_VERSION); + + ret = rdma_connect(cm_id, &conn_param); + if (ret) + rds_iwarp_conn_error(conn, "rdma_connect failed (%d)\n", ret); + +out: + /* Beware - returning non-zero tells the rdma_cm to destroy + * the cm_id. We should certainly not do it as long as we still + * "own" the cm_id. */ + if (ret) { + struct rds_iwarp_connection *ic = conn->c_transport_data; + + if (ic->i_cm_id == cm_id) + ret = 0; + } + return ret; +} + +static int rds_iwarp_cm_event_handler(struct rdma_cm_id *cm_id, + struct rdma_cm_event *event) +{ + /* this can be null in the listening path */ + struct rds_connection *conn = cm_id->context; + int ret = 0; + + rdsdebug("conn %p id %p handling event %u\n", conn, cm_id, + event->event); + + /* Prevent shutdown from tearing down the connection + * while we're executing. */ + if (conn) { + mutex_lock(&conn->c_cm_lock); + + /* If the connection is being shut down, bail out + * right away. We return 0 so cm_id doesn't get + * destroyed prematurely */ + if (atomic_read(&conn->c_state) == RDS_CONN_DISCONNECTING) { + /* Reject incoming connections while we're tearing + * down an existing one. */ + if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST) + ret = 1; + goto out; + } + } + + switch (event->event) { + case RDMA_CM_EVENT_CONNECT_REQUEST: + ret = rds_iwarp_cm_handle_connect(cm_id, event); + break; + + case RDMA_CM_EVENT_ADDR_RESOLVED: + /* XXX do we need to clean up if this fails? */ + ret = rdma_resolve_route(cm_id, + RDS_IWARP_RESOLVE_TIMEOUT_MS); + break; + + case RDMA_CM_EVENT_ROUTE_RESOLVED: + /* XXX worry about racing with listen acceptance */ + ret = rds_iwarp_cm_initiate_connect(cm_id); + break; + + case RDMA_CM_EVENT_ESTABLISHED: + rds_iwarp_connect_complete(conn, event); + break; + + case RDMA_CM_EVENT_ADDR_ERROR: + case RDMA_CM_EVENT_ROUTE_ERROR: + case RDMA_CM_EVENT_CONNECT_ERROR: + case RDMA_CM_EVENT_UNREACHABLE: + case RDMA_CM_EVENT_REJECTED: + case RDMA_CM_EVENT_DEVICE_REMOVAL: + if (conn) + rds_conn_drop(conn); + break; + + case RDMA_CM_EVENT_DISCONNECTED: + rds_conn_drop(conn); + break; + + default: + /* things like device disconnect? */ + printk(KERN_ERR "unknown event %u\n", event->event); + BUG(); + break; + } + +out: + if (conn) { + struct rds_iwarp_connection *ic = conn->c_transport_data; + + /* If we return non-zero, we must to hang on to the cm_id */ + BUG_ON(ic->i_cm_id == cm_id && ret); + + mutex_unlock(&conn->c_cm_lock); + } + + rdsdebug("id %p event %u handling ret %d\n", cm_id, event->event, ret); + + return ret; +} + +int rds_iwarp_conn_connect(struct rds_connection *conn) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct sockaddr_in src, dest; + int ret; + + /* XXX I wonder what affect the port space has */ + ic->i_cm_id = rdma_create_id(rds_iwarp_cm_event_handler, conn, + RDMA_PS_TCP); + if (IS_ERR(ic->i_cm_id)) { + ret = PTR_ERR(ic->i_cm_id); + ic->i_cm_id = NULL; + rdsdebug("rdma_create_id() failed: %d\n", ret); + goto out; + } + + rdsdebug("created cm id %p for conn %p\n", ic->i_cm_id, conn); + + src.sin_family = AF_INET; + src.sin_addr.s_addr = (__force u32)conn->c_laddr; + src.sin_port = (__force u16)htons(0); + + dest.sin_family = AF_INET; + dest.sin_addr.s_addr = (__force u32)conn->c_faddr; + dest.sin_port = (__force u16)htons(RDS_IWARP_PORT); + + ret = rdma_resolve_addr(ic->i_cm_id, (struct sockaddr *)&src, + (struct sockaddr *)&dest, + RDS_IWARP_RESOLVE_TIMEOUT_MS); + if (ret) { + rdsdebug("addr resolve failed for cm id %p: %d\n", ic->i_cm_id, + ret); + rdma_destroy_id(ic->i_cm_id); + ic->i_cm_id = NULL; + } + +out: + return ret; +} + +/* + * This is so careful about only cleaning up resources that were built up + * so that it can be called at any point during startup. In fact it + * can be called multiple times for a given connection. + */ +void rds_iwarp_conn_shutdown(struct rds_connection *conn) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + + rdsdebug("cm %p pd %p cq %p %p qp %p\n", ic->i_cm_id, + ic->i_pd, ic->i_send_cq, ic->i_recv_cq, + ic->i_cm_id ? ic->i_cm_id->qp : NULL); + + if (ic->i_cm_id) { + int err; + struct ib_device *dev = ic->i_cm_id->device; + + rdsdebug("disconnecting cm %p\n", ic->i_cm_id); + err = rdma_disconnect(ic->i_cm_id); + if (err) { + /* Actually this may happen quite frequently, when + * an outgoing connect raced with an incoming connect. + */ + printk(KERN_DEBUG "rds_iwarp_conn_shutdown: failed to disconnect," + " cm: %p err %d\n", ic->i_cm_id, err); + } + + /* NOTE: For iWARP, it is unnecessary to move the QP to error state */ + + wait_event(rds_iwarp_ring_empty_wait, + rds_iwarp_ring_empty(&ic->i_send_ring) && + rds_iwarp_ring_empty(&ic->i_recv_ring)); + + if (ic->i_send_hdrs) + ib_dma_free_coherent(dev, + ic->i_send_ring.w_nr * + sizeof(struct rds_header), + ic->i_send_hdrs, + ic->i_send_hdrs_dma); + + if (ic->i_recv_hdrs) + ib_dma_free_coherent(dev, + ic->i_recv_ring.w_nr * + sizeof(struct rds_header), + ic->i_recv_hdrs, + ic->i_recv_hdrs_dma); + + if (ic->i_ack) + ib_dma_free_coherent(dev, sizeof(struct rds_header), + ic->i_ack, ic->i_ack_dma); + + if (ic->i_sends) + rds_iwarp_send_clear_ring(ic); + if (ic->i_recvs) + rds_iwarp_recv_clear_ring(ic); + + if (ic->i_cm_id->qp) + rdma_destroy_qp(ic->i_cm_id); + if (ic->i_send_cq) + ib_destroy_cq(ic->i_send_cq); + if (ic->i_recv_cq) + ib_destroy_cq(ic->i_recv_cq); + rdma_destroy_id(ic->i_cm_id); + ic->i_cm_id = NULL; + ic->i_pd = NULL; + ic->i_mr = NULL; + ic->i_send_cq = NULL; + ic->i_recv_cq = NULL; + ic->i_send_hdrs = NULL; + ic->i_recv_hdrs = NULL; + ic->i_ack = NULL; + } + + /* Clear pending transmit */ + if (ic->i_rm) { + rds_message_put(ic->i_rm); + ic->i_rm = NULL; + } + + /* Clear the ACK state */ + clear_bit(IWARP_ACK_IN_FLIGHT, &ic->i_ack_flags); +#ifdef KERNEL_HAS_ATOMIC64 + atomic64_set(&ic->i_ack_next, 0); +#else + ic->i_ack_next = 0; +#endif + ic->i_ack_recv = 0; + + /* Clear flow control state */ + ic->i_flowctl = 0; + atomic_set(&ic->i_credits, 0); + + if (ic->i_iwarpinc) { + rds_inc_put(&ic->i_iwarpinc->ii_inc); + ic->i_iwarpinc = NULL; + } + + kfree(ic->i_sends); + ic->i_sends = NULL; + kfree(ic->i_recvs); + ic->i_recvs = NULL; +} + +int rds_iwarp_conn_alloc(struct rds_connection *conn, gfp_t gfp) +{ + struct rds_iwarp_connection *ic; + + /* XXX too lazy? */ + ic = kzalloc(sizeof(struct rds_iwarp_connection), GFP_KERNEL); + if (ic == NULL) + return -ENOMEM; + + mutex_init(&ic->i_recv_mutex); +#ifndef KERNEL_HAS_ATOMIC64 + spin_lock_init(&ic->i_ack_lock); +#endif + + /* + * rds_iwarp_conn_shutdown() waits for these to be emptied so they + * must be initialized before it can be called. + */ + rds_iwarp_ring_init(&ic->i_send_ring, rds_iwarp_sysctl_max_send_wr); + rds_iwarp_ring_init(&ic->i_recv_ring, rds_iwarp_sysctl_max_recv_wr); + + conn->c_transport_data = ic; + + rdsdebug("conn %p conn ic %p\n", conn, conn->c_transport_data); + return 0; +} + +void rds_iwarp_conn_free(void *arg) +{ + struct rds_iwarp_connection *ic = arg; + rdsdebug("ic %p\n", ic); + kfree(ic); +} + +int __init rds_iwarp_listen_init(void) +{ + struct sockaddr_in sin; + struct rdma_cm_id *cm_id; + int ret; + + cm_id = rdma_create_id(rds_iwarp_cm_event_handler, NULL, RDMA_PS_TCP); + if (IS_ERR(cm_id)) { + ret = PTR_ERR(cm_id); + printk(KERN_ERR "RDS/IWARP: failed to setup listener, " + "rdma_create_id() returned %d\n", ret); + goto out; + } + + sin.sin_family = PF_INET, + sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); + sin.sin_port = (__force u16)htons(RDS_IWARP_PORT); + + /* + * XXX I bet this binds the cm_id to a device. If we want to support + * fail-over we'll have to take this into consideration. + */ + ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin); + if (ret) { + printk(KERN_ERR "RDS/IWARP: failed to setup listener, " + "rdma_bind_addr() returned %d\n", ret); + goto out; + } + + ret = rdma_listen(cm_id, 128); + if (ret) { + printk(KERN_ERR "RDS/WARP: failed to setup listener, " + "rdma_listen() returned %d\n", ret); + goto out; + } + + rdsdebug("cm %p listening on port %u\n", cm_id, RDS_IWARP_PORT); + + rds_iwarp_listen_id = cm_id; + cm_id = NULL; +out: + if (cm_id) + rdma_destroy_id(cm_id); + return ret; +} + +void rds_iwarp_listen_stop(void) +{ + if (rds_iwarp_listen_id) { + rdsdebug("cm %p\n", rds_iwarp_listen_id); + rdma_destroy_id(rds_iwarp_listen_id); + rds_iwarp_listen_id = NULL; + } +} + +/* + * An error occurred on the connection + */ +void +__rds_iwarp_conn_error(struct rds_connection *conn, const char *fmt, ...) +{ + va_list ap; + + rds_conn_drop(conn); + + va_start(ap, fmt); + vprintk(fmt, ap); + va_end(ap); +} diff --git a/net/rds/iwarp_rdma.c b/net/rds/iwarp_rdma.c new file mode 100644 index 0000000..93b7d2b --- /dev/null +++ b/net/rds/iwarp_rdma.c @@ -0,0 +1,572 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include + +#include "rds.h" +#include "rdma.h" +#include "iwarp.h" + + +extern struct list_head rds_iwarp_devices; + +/* + * This is stored as mr->r_trans_private. + */ +struct rds_iwarp_mr { + struct rds_iwarp_device *device; + struct rds_iwarp_mr_pool *pool; + struct ib_fast_reg_page_list *page_list; + struct ib_mr *fastreg_mr; + struct list_head list; + unsigned int remap_count; + struct scatterlist * sg; + unsigned int sg_len; + u64 * dma; + int sg_dma_len; + u8 rkey_cnt; + struct ib_send_wr invalidate_wr; + struct ib_send_wr fastreg_wr; +}; + +/* + * Our own little FMR pool + */ +struct rds_iwarp_mr_pool { + struct mutex flush_lock; /* serialize fmr invalidate */ + struct work_struct flush_worker; /* flush worker */ + + spinlock_t list_lock; /* protect variables below */ + atomic_t item_count; /* total # of MRs */ + atomic_t dirty_count; /* # dirty of MRs */ + struct list_head drop_list; /* MRs that have reached their max_maps limit */ + struct list_head free_list; /* unused MRs */ + struct list_head clean_list; /* unused & unamapped MRs */ + atomic_t free_pinned; /* memory pinned by free MRs */ + unsigned long max_items; + unsigned long max_items_soft; + unsigned long max_free_pinned; + struct ib_fmr_attr fmr_attr; +}; + +static int rds_iwarp_flush_mr_pool(struct rds_iwarp_mr_pool *pool, int free_all); +static void rds_iwarp_teardown_mr(struct rds_iwarp_mr *ibmr); +static void rds_iwarp_mr_pool_flush_worker(struct work_struct *work); + +struct rds_iwarp_device* iwarp_get_device(__be32 ipaddr) +{ + struct rds_iwarp_device *rds_iwarpdev; + struct rds_iwarp_ipaddr *i_ipaddr; + + list_for_each_entry(rds_iwarpdev, &rds_iwarp_devices, list) { + spin_lock_irq(&rds_iwarpdev->spinlock); + list_for_each_entry(i_ipaddr, &rds_iwarpdev->ipaddr_list, list) { + if (i_ipaddr->ipaddr == ipaddr) { + spin_unlock_irq(&rds_iwarpdev->spinlock); + return rds_iwarpdev; + } + } + spin_unlock_irq(&rds_iwarpdev->spinlock); + } + + return NULL; +} + +struct rds_iwarp_mr_pool *rds_iwarp_create_mr_pool(struct rds_iwarp_device *rds_iwarpdev) +{ + struct rds_iwarp_mr_pool *pool; + + pool = kzalloc(sizeof(*pool), GFP_KERNEL); + if (!pool) + return ERR_PTR(-ENOMEM); + + INIT_LIST_HEAD(&pool->free_list); + INIT_LIST_HEAD(&pool->drop_list); + INIT_LIST_HEAD(&pool->clean_list); + mutex_init(&pool->flush_lock); + spin_lock_init(&pool->list_lock); + INIT_WORK(&pool->flush_worker, rds_iwarp_mr_pool_flush_worker); + + /* We never allow more than max_items MRs to be allocated. + * When we exceed more than max_items_soft, we start freeing + * items more aggressively. + * Make sure that max_items > max_items_soft > max_items / 2 + */ + pool->max_items_soft = rds_iwarpdev->max_fmrs * 3 / 4; + pool->max_items = rds_iwarpdev->max_fmrs; + + return pool; +} + +void rds_iwarp_destroy_mr_pool(struct rds_iwarp_mr_pool *pool) +{ + flush_workqueue(rds_wq); + rds_iwarp_flush_mr_pool(pool, 1); + BUG_ON(atomic_read(&pool->item_count)); + BUG_ON(atomic_read(&pool->free_pinned)); + kfree(pool); +} + +static inline struct rds_iwarp_mr *rds_iwarp_reuse_fmr(struct rds_iwarp_mr_pool *pool) +{ + struct rds_iwarp_mr *ibmr = NULL; + unsigned long flags; + + spin_lock_irqsave(&pool->list_lock, flags); + if (!list_empty(&pool->clean_list)) { + ibmr = list_entry(pool->clean_list.next, struct rds_iwarp_mr, list); + list_del_init(&ibmr->list); + } + spin_unlock_irqrestore(&pool->list_lock, flags); + + return ibmr; +} + +static struct rds_iwarp_mr *rds_iwarp_alloc_fmr(struct rds_iwarp_device *rds_iwarpdev) +{ + struct rds_iwarp_mr_pool *pool = rds_iwarpdev->mr_pool; + struct rds_iwarp_mr *ibmr = NULL; + int err = 0, iter = 0; + + while (1) { + if ((ibmr = rds_iwarp_reuse_fmr(pool)) != NULL) + return ibmr; + + /* No clean MRs - now we have the choice of either + * allocating a fresh MR up to the limit imposed by the + * driver, or flush any dirty unused MRs. + * We try to avoid stalling in the send path if possible, + * so we allocate as long as we're allowed to. + * + * We're fussy with enforcing the FMR limit, though. If the driver + * tells us we can't use more than N fmrs, we shouldn't start + * arguing with it */ + if (atomic_inc_return(&pool->item_count) <= pool->max_items) + break; + + atomic_dec(&pool->item_count); + + if (++iter > 2) + return ERR_PTR(-EAGAIN); + + /* We do have some empty MRs. Flush them out. */ + rds_iwarp_stats_inc(s_iwarp_rdma_mr_pool_wait); + rds_iwarp_flush_mr_pool(pool, 0); + } + + ibmr = kzalloc(sizeof(*ibmr), GFP_KERNEL); + if (!ibmr) { + err = -ENOMEM; + goto out_no_cigar; + } + + ibmr->page_list = ib_alloc_fast_reg_page_list(rds_iwarpdev->pd->device, fast_mr_message_size); + if (IS_ERR(ibmr->page_list)) { + err = PTR_ERR(ibmr->page_list); + ibmr->page_list = NULL; + printk(KERN_WARNING "RDS/IWARP: ib_alloc_fast_reg_page_list failed (err=%d)\n", err); + goto out_no_cigar; + } + + ibmr->fastreg_mr = ib_alloc_fast_reg_mr(rds_iwarpdev->pd, ibmr->page_list->max_page_list_len); + if (IS_ERR(ibmr->page_list)) { + err = PTR_ERR(ibmr->fastreg_mr); + ibmr->fastreg_mr = NULL; + printk(KERN_WARNING "RDS/IWARP: ib_alloc_fast_reg_mr failed (err=%d)\n", err); + goto out_no_cigar; + } + + ibmr->rkey_cnt = 0; + + rds_iwarp_stats_inc(s_iwarp_rdma_mr_alloc); + return ibmr; + +out_no_cigar: + if (ibmr) { + if (ibmr->fastreg_mr) + ib_dereg_mr(ibmr->fastreg_mr); + kfree(ibmr); + } + atomic_dec(&pool->item_count); + return ERR_PTR(err); +} + +static int rds_iwarp_map_fmr(struct rds_iwarp_device *rds_iwarpdev, struct rds_iwarp_mr *ibmr, + struct scatterlist *sg, unsigned int nents, struct ib_qp *qp) +{ + struct ib_device *dev = rds_iwarpdev->dev; + struct scatterlist *scat = sg; + u32 len; + int page_cnt, sg_dma_len; + int i, j; + int ret; + struct ib_send_wr *bad_wr; + struct ib_mr *mr; + + sg_dma_len = ib_dma_map_sg(dev, sg, nents, + DMA_BIDIRECTIONAL); + if (unlikely(!sg_dma_len)) { + printk(KERN_WARNING "RDS/IWARP: dma_map_sg failed!\n"); + return -EBUSY; + } + + len = 0; + page_cnt = 0; + + for (i = 0; i < sg_dma_len; ++i) { + unsigned int dma_len = ib_sg_dma_len(dev, &scat[i]); + u64 dma_addr = ib_sg_dma_address(dev, &scat[i]); + + if (dma_addr & ~rds_iwarpdev->fmr_page_mask) { + if (i > 0) + return -EINVAL; + else + ++page_cnt; + } + if ((dma_addr + dma_len) & ~rds_iwarpdev->fmr_page_mask) { + if (i < sg_dma_len - 1) + return -EINVAL; + else + ++page_cnt; + } + + len += dma_len; + } + + page_cnt += len >> rds_iwarpdev->fmr_page_shift; + if (page_cnt > fast_mr_message_size) + return -EINVAL; + + /* + * Perform 2 WRs for the fast_reg_mr's and Chain them together. The + * first WR is used to invalidate the old rkey, and the second WR is + * used to define the new fast_reg_mr request. Each individual page + * in the sg list is added to the fast reg page list and placed + * inside the fast_reg_mr WR. The key used is a rolling 8bit + * counter, which should guarantee uniqueness. + */ + mr = ibmr->fastreg_mr; + memset(&ibmr->invalidate_wr, 0, sizeof(ibmr->invalidate_wr)); + ibmr->invalidate_wr.ex.invalidate_rkey = mr->rkey; + ibmr->invalidate_wr.opcode = IB_WR_LOCAL_INV; + + ib_update_fast_reg_key(mr, ibmr->rkey_cnt++); + memset(&ibmr->fastreg_wr, 0, sizeof(ibmr->fastreg_wr)); + ibmr->fastreg_wr.opcode = IB_WR_FAST_REG_MR; + ibmr->fastreg_wr.wr.fast_reg.length = len; + ibmr->fastreg_wr.wr.fast_reg.rkey = mr->rkey; + ibmr->fastreg_wr.wr.fast_reg.page_list = ibmr->page_list; + ibmr->fastreg_wr.wr.fast_reg.page_list_len = sg_dma_len; + ibmr->fastreg_wr.wr.fast_reg.page_shift = rds_iwarpdev->fmr_page_shift; + + ibmr->invalidate_wr.next = &ibmr->fastreg_wr; + + ibmr->fastreg_wr.wr.fast_reg.access_flags = IB_ACCESS_LOCAL_WRITE | + IB_ACCESS_REMOTE_READ | + IB_ACCESS_REMOTE_WRITE; + ibmr->fastreg_wr.wr.fast_reg.iova_start = 0; + + rdsdebug("%d old rkey = 0x%x....new rkey = 0x%x\n", + ibmr->rkey_cnt, + ibmr->invalidate_wr.ex.invalidate_rkey, + ibmr->fastreg_wr.wr.fast_reg.rkey); + rdsdebug("page shift = %d length = %d flags 0x%x\n", + ibmr->fastreg_wr.wr.fast_reg.page_shift, + ibmr->fastreg_wr.wr.fast_reg.length, + ibmr->fastreg_wr.wr.fast_reg.access_flags); + rdsdebug("page list %#lx page list len %d\n", + ibmr->fastreg_wr.wr.fast_reg.page_list, + ibmr->fastreg_wr.wr.fast_reg.page_list_len); + rdsdebug("iova start %lx\n", ibmr->fastreg_wr.wr.fast_reg.iova_start); + + page_cnt = 0; + for (i = 0; i < sg_dma_len; ++i) { + unsigned int dma_len = ib_sg_dma_len(dev, &scat[i]); + u64 dma_addr = ib_sg_dma_address(dev, &scat[i]); + + for (j = 0; j < dma_len; j += rds_iwarpdev->fmr_page_size) { + ibmr->page_list->page_list[page_cnt++] = + (dma_addr & rds_iwarpdev->fmr_page_mask) + j; + rdsdebug("page list entry %lx\n", + ibmr->page_list->page_list[page_cnt-1]); + } + } + + if (qp) { + ret = ib_post_send(qp, &ibmr->fastreg_wr, &bad_wr); + if (ret) { + printk(KERN_ERR "post send error %d\n", ret); + return -EINVAL; + } + } + + ibmr->sg = scat; + ibmr->sg_len = nents; + ibmr->sg_dma_len = sg_dma_len; + ibmr->remap_count++; + + rds_iwarp_stats_inc(s_iwarp_rdma_mr_used); + ret = 0; + + return ret; +} + +void rds_iwarp_sync_mr(void *trans_private, int direction) +{ + struct rds_iwarp_mr *ibmr = trans_private; + struct rds_iwarp_device *rds_iwarpdev = ibmr->device; + + switch (direction) { + case DMA_FROM_DEVICE: + ib_dma_sync_sg_for_cpu(rds_iwarpdev->dev, ibmr->sg, + ibmr->sg_dma_len, DMA_BIDIRECTIONAL); + break; + case DMA_TO_DEVICE: + ib_dma_sync_sg_for_device(rds_iwarpdev->dev, ibmr->sg, + ibmr->sg_dma_len, DMA_BIDIRECTIONAL); + break; + } +} + +static void __rds_iwarp_teardown_mr(struct rds_iwarp_mr *ibmr) +{ + struct rds_iwarp_device *rds_iwarpdev = ibmr->device; + + if (ibmr->sg_dma_len) { + ib_dma_unmap_sg(rds_iwarpdev->dev, + ibmr->sg, ibmr->sg_len, + DMA_BIDIRECTIONAL); + ibmr->sg_dma_len = 0; + } + + /* Release the s/g list */ + if (ibmr->sg_len) { + unsigned int i; + + for (i = 0; i < ibmr->sg_len; ++i) { + struct page *page = sg_page(&ibmr->sg[i]); + + /* FIXME we need a way to tell a r/w MR + * from a r/o MR */ + set_page_dirty(page); + put_page(page); + } + kfree(ibmr->sg); + + ibmr->sg = NULL; + ibmr->sg_len = 0; + } +} + +void rds_iwarp_teardown_mr(struct rds_iwarp_mr *ibmr) +{ + unsigned int pinned = ibmr->sg_len; + + __rds_iwarp_teardown_mr(ibmr); + if (pinned) { + struct rds_iwarp_device *rds_iwarpdev = ibmr->device; + struct rds_iwarp_mr_pool *pool = rds_iwarpdev->mr_pool; + + atomic_sub(pinned, &pool->free_pinned); + } +} + +static inline unsigned int rds_iwarp_flush_goal(struct rds_iwarp_mr_pool *pool, int free_all) +{ + unsigned int item_count; + + item_count = atomic_read(&pool->item_count); + if (free_all) + return item_count; +#if 0 + if (item_count > pool->max_items_soft) + return item_count - pool->max_items / 2; +#endif + return 0; +} + +/* + * Flush our pool of MRs. + * At a minimum, all currently unused MRs are unmapped. + * If the number of MRs allocated exceeds the limit, we also try + * to free as many MRs as needed to get back to this limit. + */ +int rds_iwarp_flush_mr_pool(struct rds_iwarp_mr_pool *pool, int free_all) +{ + struct rds_iwarp_mr *ibmr, *next; + LIST_HEAD(unmap_list); + LIST_HEAD(fmr_list); + unsigned long unpinned = 0; + unsigned long flags; + unsigned int nfreed = 0, ncleaned = 0, free_goal; + int ret = 0; + + rds_iwarp_stats_inc(s_iwarp_rdma_mr_pool_flush); + + mutex_lock(&pool->flush_lock); + + spin_lock_irqsave(&pool->list_lock, flags); + /* Get the list of all MRs to be dropped. Ordering matters - + * we want to put drop_list ahead of free_list. */ + list_splice_init(&pool->free_list, &unmap_list); + list_splice_init(&pool->drop_list, &unmap_list); + if (free_all) + list_splice_init(&pool->clean_list, &unmap_list); + spin_unlock_irqrestore(&pool->list_lock, flags); + + free_goal = rds_iwarp_flush_goal(pool, free_all); + + if (list_empty(&unmap_list)) + goto out; + + ib_dereg_mr(ibmr->fastreg_mr); + + /* Now we can destroy the DMA mapping and unpin any pages */ + list_for_each_entry_safe(ibmr, next, &unmap_list, list) { + unpinned += ibmr->sg_len; + __rds_iwarp_teardown_mr(ibmr); + if (nfreed < free_goal || ibmr->remap_count >= pool->fmr_attr.max_maps) { + rds_iwarp_stats_inc(s_iwarp_rdma_mr_free); + list_del(&ibmr->list); + kfree(ibmr); + nfreed++; + } + ncleaned++; + } + + spin_lock_irqsave(&pool->list_lock, flags); + list_splice(&unmap_list, &pool->clean_list); + spin_unlock_irqrestore(&pool->list_lock, flags); + + atomic_sub(unpinned, &pool->free_pinned); + atomic_sub(ncleaned, &pool->dirty_count); + atomic_sub(nfreed, &pool->item_count); + +out: + mutex_unlock(&pool->flush_lock); + return ret; +} + +void rds_iwarp_mr_pool_flush_worker(struct work_struct *work) +{ + struct rds_iwarp_mr_pool *pool = container_of(work, struct rds_iwarp_mr_pool, flush_worker); + + rds_iwarp_flush_mr_pool(pool, 0); +} + +void rds_iwarp_free_mr(void *trans_private, int invalidate) +{ + struct rds_iwarp_mr *ibmr = trans_private; + struct rds_iwarp_device *rds_iwarpdev = ibmr->device; + struct rds_iwarp_mr_pool *pool = rds_iwarpdev->mr_pool; + unsigned long flags; + + rdsdebug("RDS/IWARP: free_mr nents %u\n", ibmr->sg_len); + + /* Return it to the pool's free list */ + spin_lock_irqsave(&pool->list_lock, flags); + if (ibmr->remap_count >= pool->fmr_attr.max_maps) { + list_add(&ibmr->list, &pool->drop_list); + } else { + list_add(&ibmr->list, &pool->free_list); + } + atomic_add(ibmr->sg_len, &pool->free_pinned); + atomic_inc(&pool->dirty_count); + spin_unlock_irqrestore(&pool->list_lock, flags); + + /* If we've pinned too many pages, request a flush */ + if (atomic_read(&pool->free_pinned) >= pool->max_free_pinned + || atomic_read(&pool->dirty_count) >= pool->max_items / 10) + queue_work(rds_wq, &pool->flush_worker); + + if (invalidate) { + if (likely(!in_interrupt())) { + rds_iwarp_flush_mr_pool(pool, 0); + } else { + /* We get here if the user created a MR marked + * as use_once and invalidate at the same time. */ + queue_work(rds_wq, &pool->flush_worker); + } + } +} + +void rds_iwarp_flush_mrs(void) +{ + struct rds_iwarp_device *rds_iwarpdev; + + list_for_each_entry(rds_iwarpdev, &rds_iwarp_devices, list) { + struct rds_iwarp_mr_pool *pool = rds_iwarpdev->mr_pool; + + if (pool) + rds_iwarp_flush_mr_pool(pool, 0); + } +} + +void *rds_iwarp_get_mr(struct scatterlist *sg, unsigned long nents, + struct rds_sock *rs, u32 *key_ret) +{ + struct rds_iwarp_device *rds_iwarpdev; + struct rds_iwarp_mr *ibmr = NULL; + __be32 ip_addr = rs->rs_bound_addr; + int ret; + + rds_iwarpdev = iwarp_get_device(ip_addr); + if (!rds_iwarpdev) { + ret = -ENODEV; + goto out; + } + + if (!rds_iwarpdev->mr_pool) { + ret = -ENODEV; + goto out; + } + + ibmr = rds_iwarp_alloc_fmr(rds_iwarpdev); + if (IS_ERR(ibmr)) + return ibmr; + + ret = rds_iwarp_map_fmr(rds_iwarpdev, ibmr, sg, nents, ((struct rds_iwarp_connection *)rs->rs_conn->c_transport_data)->i_cm_id->qp); + if (ret == 0) + *key_ret = ibmr->fastreg_mr->rkey; + else + printk(KERN_WARNING "RDS/IWARP: map_fmr failed (errno=%d)\n", ret); + + ibmr->device = rds_iwarpdev; + + out: + if (ret) { + if (ibmr) + rds_iwarp_free_mr(ibmr, 0); + ibmr = ERR_PTR(ret); + } + return ibmr; +} diff --git a/net/rds/iwarp_rds.h b/net/rds/iwarp_rds.h new file mode 100644 index 0000000..fe11c92 --- /dev/null +++ b/net/rds/iwarp_rds.h @@ -0,0 +1,240 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#ifndef IB_RDS_H +#define IB_RDS_H + +#include + +/* These sparse annotated types shouldn't be in any user + * visible header file. We should clean this up rather + * than kludging around them. */ +#ifndef __KERNEL__ +#define __be16 u_int16_t +#define __be32 u_int32_t +#define __be64 u_int64_t +#endif + +#define RDS_IB_ABI_VERSION 0x301 + +/* + * setsockopt/getsockopt for SOL_RDS + */ +#define RDS_CANCEL_SENT_TO 1 +#define RDS_GET_MR 2 +#define RDS_FREE_MR 3 +/* deprecated: RDS_BARRIER 4 */ +#define RDS_RECVERR 5 +#define RDS_CONG_MONITOR 6 + +/* + * Control message types for SOL_RDS. + * + * CMSG_RDMA_ARGS (sendmsg) + * Request a RDMA transfer to/from the specified + * memory ranges. + * The cmsg_data is a struct rds_rdma_args. + * RDS_CMSG_RDMA_DEST (recvmsg, sendmsg) + * Kernel informs application about intended + * source/destination of a RDMA transfer + * RDS_CMSG_RDMA_MAP (sendmsg) + * Application asks kernel to map the given + * memory range into a IB MR, and send the + * R_Key along in an RDS extension header. + * The cmsg_data is a struct rds_get_mr_args, + * the same as for the GET_MR setsockopt. + * RDS_CMSG_RDMA_STATUS (recvmsg) + * Returns the status of a completed RDMA operation. + */ +#define RDS_CMSG_RDMA_ARGS 1 +#define RDS_CMSG_RDMA_DEST 2 +#define RDS_CMSG_RDMA_MAP 3 +#define RDS_CMSG_RDMA_STATUS 4 +#define RDS_CMSG_CONG_UPDATE 5 + +#define RDS_INFO_COUNTERS 10000 +#define RDS_INFO_CONNECTIONS 10001 +/* 10002 aka RDS_INFO_FLOWS is deprecated */ +#define RDS_INFO_SEND_MESSAGES 10003 +#define RDS_INFO_RETRANS_MESSAGES 10004 +#define RDS_INFO_RECV_MESSAGES 10005 +#define RDS_INFO_SOCKETS 10006 +#define RDS_INFO_TCP_SOCKETS 10007 + +struct rds_info_counter { + u_int8_t name[32]; + u_int64_t value; +} __attribute__((packed)); + +#define RDS_INFO_CONNECTION_FLAG_SENDING 0x01 +#define RDS_INFO_CONNECTION_FLAG_CONNECTING 0x02 +#define RDS_INFO_CONNECTION_FLAG_CONNECTED 0x04 + +struct rds_info_connection { + u_int64_t next_tx_seq; + u_int64_t next_rx_seq; + __be32 laddr; + __be32 faddr; + u_int8_t transport[15]; /* null term ascii */ + u_int8_t flags; +} __attribute__((packed)); + +struct rds_info_flow { + __be32 laddr; + __be32 faddr; + u_int32_t bytes; + __be16 lport; + __be16 fport; +} __attribute__((packed)); + +#define RDS_INFO_MESSAGE_FLAG_ACK 0x01 +#define RDS_INFO_MESSAGE_FLAG_FAST_ACK 0x02 + +struct rds_info_message { + u_int64_t seq; + u_int32_t len; + __be32 laddr; + __be32 faddr; + __be16 lport; + __be16 fport; + u_int8_t flags; +} __attribute__((packed)); + +struct rds_info_socket { + u_int32_t sndbuf; + __be32 bound_addr; + __be32 connected_addr; + __be16 bound_port; + __be16 connected_port; + u_int32_t rcvbuf; +} __attribute__((packed)); + +struct rds_info_tcp_socket { + __be32 local_addr; + __be16 local_port; + __be32 peer_addr; + __be16 peer_port; + u_int64_t hdr_rem; + u_int64_t data_rem; + u_int32_t last_sent_nxt; + u_int32_t last_expected_una; + u_int32_t last_seen_una; +} __attribute__((packed)); + +/* + * Congestion monitoring. + * Congestion control in RDS happens at the host connection + * level by exchanging a bitmap marking congested ports. + * By default, a process sleeping in poll() is always woken + * up when the congestion map is updated. + * With explicit monitoring, an application can have more + * fine-grained control. + * The application installs a 64bit mask value in the socket, + * where each bit corresponds to a group of ports. + * When a congestion update arrives, RDS checks the set of + * ports that are now uncongested against the list bit mask + * installed in the socket, and if they overlap, we queue a + * cong_notification on the socket. + * + * To install the congestion monitor bitmask, use RDS_CONG_MONITOR + * with the 64bit mask. + * Congestion updates are received via RDS_CMSG_CONG_UPDATE + * control messages. + * + * The correspondence between bits and ports is + * 1 << (portnum % 64) + */ +#define RDS_CONG_MONITOR_SIZE 64 +#define RDS_CONG_MONITOR_BIT(port) (((unsigned int) port) % RDS_CONG_MONITOR_SIZE) +#define RDS_CONG_MONITOR_MASK(port) (1ULL << RDS_CONG_MONITOR_BIT(port)) + +/* + * RDMA related types + */ + +/* + * This encapsulates a remote memory location. + * In the current implementation, it contains the R_Key + * of the remote memory region, and the offset into it + * (so that the application does not have to worry about + * alignment). + */ +typedef u_int64_t rds_rdma_cookie_t; + +struct rds_iovec { + u_int64_t addr; + u_int64_t bytes; +}; + +struct rds_get_mr_args { + struct rds_iovec vec; + u_int64_t cookie_addr; + uint64_t flags; +}; + +struct rds_free_mr_args { + rds_rdma_cookie_t cookie; + u_int64_t flags; +}; + +struct rds_rdma_args { + rds_rdma_cookie_t cookie; + struct rds_iovec remote_vec; + u_int64_t local_vec_addr; + u_int64_t nr_local; + u_int64_t flags; + u_int64_t user_token; +}; + +struct rds_rdma_notify { + u_int64_t user_token; + int32_t status; +}; + +#define RDS_RDMA_SUCCESS 0 +#define RDS_RDMA_REMOTE_ERROR 1 +#define RDS_RDMA_CANCELED 2 +#define RDS_RDMA_DROPPED 3 +#define RDS_RDMA_OTHER_ERROR 4 + +/* + * Common set of flags for all RDMA related structs + */ +#define RDS_RDMA_READWRITE 0x0001 +#define RDS_RDMA_FENCE 0x0002 /* use FENCE for immediate send */ +#define RDS_RDMA_INVALIDATE 0x0004 /* invalidate R_Key after freeing MR */ +#define RDS_RDMA_USE_ONCE 0x0008 /* free MR after use */ +#define RDS_RDMA_DONTWAIT 0x0010 /* Don't wait in SET_BARRIER */ +#define RDS_RDMA_NOTIFY_ME 0x0020 /* Notify when operation completes */ + +#endif /* IB_RDS_H */ diff --git a/net/rds/iwarp_recv.c b/net/rds/iwarp_recv.c new file mode 100644 index 0000000..778298d --- /dev/null +++ b/net/rds/iwarp_recv.c @@ -0,0 +1,968 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include +#include +#include +#include + +#include "rds.h" +#include "iwarp.h" + +static struct kmem_cache *rds_iwarp_incoming_slab; +static struct kmem_cache *rds_iwarp_frag_slab; +static atomic_t rds_iwarp_allocation = ATOMIC_INIT(0); + +static void rds_iwarp_frag_drop_page(struct rds_page_frag *frag) +{ + rdsdebug("frag %p page %p\n", frag, frag->f_page); + __free_page(frag->f_page); + frag->f_page = NULL; +} + +static void rds_iwarp_frag_free(struct rds_page_frag *frag) +{ + rdsdebug("frag %p page %p\n", frag, frag->f_page); + BUG_ON(frag->f_page != NULL); + kmem_cache_free(rds_iwarp_frag_slab, frag); +} + +/* + * We map a page at a time. Its fragments are posted in order. This + * is called in fragment order as the fragments get send completion events. + * Only the last frag in the page performs the unmapping. + * + * It's OK for ring cleanup to call this in whatever order it likes because + * DMA is not in flight and so we can unmap while other ring entries still + * hold page references in their frags. + */ +static void rds_iwarp_recv_unmap_page(struct rds_iwarp_connection *ic, + struct rds_iwarp_recv_work *recv) +{ + struct rds_page_frag *frag = recv->r_frag; + + rdsdebug("recv %p frag %p page %p\n", recv, frag, frag->f_page); + if (frag->f_mapped) + ib_dma_unmap_page(ic->i_cm_id->device, + frag->f_mapped, + RDS_FRAG_SIZE, DMA_FROM_DEVICE); + frag->f_mapped = 0; +} + +void rds_iwarp_recv_init_ring(struct rds_iwarp_connection *ic) +{ + struct rds_iwarp_recv_work *recv; + u32 i; + + for(i = 0, recv = ic->i_recvs; i < ic->i_recv_ring.w_nr; i++, recv++) { + struct ib_sge *sge; + + recv->r_iwarpinc = NULL; + recv->r_frag = NULL; + + recv->r_wr.next = NULL; + recv->r_wr.wr_id = i; + recv->r_wr.sg_list = recv->r_sge; + recv->r_wr.num_sge = RDS_IWARP_RECV_SGE; + + sge = rds_iwarp_data_sge(ic, recv->r_sge); + sge->addr = 0; + sge->length = RDS_FRAG_SIZE; + sge->lkey = ic->i_mr->lkey; + + sge = rds_iwarp_header_sge(ic, recv->r_sge); + sge->addr = ic->i_recv_hdrs_dma + (i * sizeof(struct rds_header)); + sge->length = sizeof(struct rds_header); + sge->lkey = ic->i_mr->lkey; + } +} + +static void rds_iwarp_recv_clear_one(struct rds_iwarp_connection *ic, + struct rds_iwarp_recv_work *recv) +{ + if (recv->r_iwarpinc) { + rds_inc_put(&recv->r_iwarpinc->ii_inc); + recv->r_iwarpinc = NULL; + } + if (recv->r_frag) { + rds_iwarp_recv_unmap_page(ic, recv); + if (recv->r_frag->f_page) + rds_iwarp_frag_drop_page(recv->r_frag); + rds_iwarp_frag_free(recv->r_frag); + recv->r_frag = NULL; + } +} + +void rds_iwarp_recv_clear_ring(struct rds_iwarp_connection *ic) +{ + u32 i; + + for(i = 0; i < ic->i_recv_ring.w_nr; i++) + rds_iwarp_recv_clear_one(ic, &ic->i_recvs[i]); + + if (ic->i_frag.f_page) + rds_iwarp_frag_drop_page(&ic->i_frag); +} + +static int rds_iwarp_recv_refill_one(struct rds_connection *conn, + struct rds_iwarp_recv_work *recv, + gfp_t kptr_gfp, gfp_t page_gfp) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + dma_addr_t dma_addr; + struct ib_sge *sge; + int ret = -ENOMEM; + + if (recv->r_iwarpinc == NULL) { + if (atomic_read(&rds_iwarp_allocation) >= rds_iwarp_sysctl_max_recv_allocation) { + rds_iwarp_stats_inc(s_iwarp_rx_alloc_limit); + goto out; + } + recv->r_iwarpinc = kmem_cache_alloc(rds_iwarp_incoming_slab, + kptr_gfp); + if (recv->r_iwarpinc == NULL) + goto out; + atomic_inc(&rds_iwarp_allocation); + INIT_LIST_HEAD(&recv->r_iwarpinc->ii_frags); + rds_inc_init(&recv->r_iwarpinc->ii_inc, conn, conn->c_faddr); + } + + if (recv->r_frag == NULL) { + recv->r_frag = kmem_cache_alloc(rds_iwarp_frag_slab, kptr_gfp); + if (recv->r_frag == NULL) + goto out; + INIT_LIST_HEAD(&recv->r_frag->f_item); + recv->r_frag->f_page = NULL; + } + + if (ic->i_frag.f_page == NULL) { + ic->i_frag.f_page = alloc_page(page_gfp); + if (ic->i_frag.f_page == NULL) + goto out; + ic->i_frag.f_offset = 0; + } + + dma_addr = ib_dma_map_page(ic->i_cm_id->device, + ic->i_frag.f_page, + ic->i_frag.f_offset, + RDS_FRAG_SIZE, + DMA_FROM_DEVICE); + if (ib_dma_mapping_error(ic->i_cm_id->device, dma_addr)) + goto out; + + /* + * Once we get the RDS_PAGE_LAST_OFF frag then rds_iwarp_frag_unmap() + * must be called on this recv. This happens as completions hit + * in order or on connection shutdown. + */ + recv->r_frag->f_page = ic->i_frag.f_page; + recv->r_frag->f_offset = ic->i_frag.f_offset; + recv->r_frag->f_mapped = dma_addr; + + sge = rds_iwarp_data_sge(ic, recv->r_sge); + sge->addr = dma_addr; + sge->length = RDS_FRAG_SIZE; + + sge = rds_iwarp_header_sge(ic, recv->r_sge); + sge->addr = ic->i_recv_hdrs_dma + (recv - ic->i_recvs) * sizeof(struct rds_header); + sge->length = sizeof(struct rds_header); + + get_page(recv->r_frag->f_page); + + if (ic->i_frag.f_offset < RDS_PAGE_LAST_OFF) { + ic->i_frag.f_offset += RDS_FRAG_SIZE; + } else { + put_page(ic->i_frag.f_page); + ic->i_frag.f_page = NULL; + ic->i_frag.f_offset = 0; + } + + ret = 0; +out: + return ret; +} + +/* + * This tries to allocate and post unused work requests after making sure that + * they have all the allocations they need to queue received fragments into + * sockets. The i_recv_mutex is held here so that ring_alloc and _unalloc + * pairs don't go unmatched. + * + * -1 is returned if posting fails due to temporary resource exhaustion. + */ +int rds_iwarp_recv_refill(struct rds_connection *conn, gfp_t kptr_gfp, + gfp_t page_gfp, int prefill) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct rds_iwarp_recv_work *recv; + struct ib_recv_wr *failed_wr; + unsigned int posted = 0; + int ret = 0; + u32 pos; + + while ((prefill || rds_conn_up(conn)) + && rds_iwarp_ring_alloc(&ic->i_recv_ring, 1, &pos)) { + if (pos >= ic->i_recv_ring.w_nr) { + printk(KERN_NOTICE "Argh - ring alloc returned pos=%u\n", + pos); + ret = -EINVAL; + break; + } + + recv = &ic->i_recvs[pos]; + ret = rds_iwarp_recv_refill_one(conn, recv, kptr_gfp, page_gfp); + if (ret) { + ret = -1; + break; + } + + /* XXX when can this fail? */ + ret = ib_post_recv(ic->i_cm_id->qp, &recv->r_wr, &failed_wr); + rdsdebug("recv %p ibinc %p page %p addr %lu ret %d\n", recv, + recv->r_iwarpinc, recv->r_frag->f_page, + (long) recv->r_frag->f_mapped, ret); + if (ret) { + rds_iwarp_conn_error(conn, "recv post on " + "%u.%u.%u.%u returned %d, disconnecting and " + "reconnecting\n", NIPQUAD(conn->c_faddr), + ret); + ret = -1; + break; + } + + posted++; + } + + /* We're doing flow control - update the window. */ + if (ic->i_flowctl && posted) + rds_iwarp_advertise_credits(conn, posted); + + if (ret) + rds_iwarp_ring_unalloc(&ic->i_recv_ring, 1); + return ret; +} + +void rds_iwarp_inc_purge(struct rds_incoming *inc) +{ + struct rds_iwarp_incoming *ibinc; + struct rds_page_frag *frag; + struct rds_page_frag *pos; + + ibinc = container_of(inc, struct rds_iwarp_incoming, ii_inc); + rdsdebug("purging ibinc %p inc %p\n", ibinc, inc); + + list_for_each_entry_safe(frag, pos, &ibinc->ii_frags, f_item) { + list_del_init(&frag->f_item); + rds_iwarp_frag_drop_page(frag); + rds_iwarp_frag_free(frag); + } +} + +void rds_iwarp_inc_free(struct rds_incoming *inc) +{ + struct rds_iwarp_incoming *ibinc; + + ibinc = container_of(inc, struct rds_iwarp_incoming, ii_inc); + + rds_iwarp_inc_purge(inc); + rdsdebug("freeing ibinc %p inc %p\n", ibinc, inc); + BUG_ON(!list_empty(&ibinc->ii_frags)); + kmem_cache_free(rds_iwarp_incoming_slab, ibinc); + atomic_dec(&rds_iwarp_allocation); + BUG_ON(atomic_read(&rds_iwarp_allocation) < 0); +} + +int rds_iwarp_inc_copy_to_user(struct rds_incoming *inc, struct iovec *first_iov, + size_t size) +{ + struct rds_iwarp_incoming *ibinc; + struct rds_page_frag *frag; + struct iovec *iov = first_iov; + unsigned long to_copy; + unsigned long frag_off = 0; + unsigned long iov_off = 0; + int copied = 0; + int ret; + u32 len; + + ibinc = container_of(inc, struct rds_iwarp_incoming, ii_inc); + frag = list_entry(ibinc->ii_frags.next, struct rds_page_frag, f_item); + len = be32_to_cpu(inc->i_hdr.h_len); + + while (copied < size && copied < len) { + if (frag_off == RDS_FRAG_SIZE) { + frag = list_entry(frag->f_item.next, + struct rds_page_frag, f_item); + frag_off = 0; + } + while (iov_off == iov->iov_len) { + iov_off = 0; + iov++; + } + + to_copy = min(iov->iov_len - iov_off, RDS_FRAG_SIZE - frag_off); + to_copy = min_t(size_t, to_copy, size - copied); + to_copy = min_t(unsigned long, to_copy, len - copied); + + rdsdebug("%lu bytes to user [%p, %zu] + %lu from frag " + "[%p, %lu] + %lu\n", + to_copy, iov->iov_base, iov->iov_len, iov_off, + frag->f_page, frag->f_offset, frag_off); + + /* XXX needs + offset for multiple recvs per page */ + ret = rds_page_copy_to_user(frag->f_page, + frag->f_offset + frag_off, + iov->iov_base + iov_off, + to_copy); + if (ret) { + copied = ret; + break; + } + + iov_off += to_copy; + frag_off += to_copy; + copied += to_copy; + } + + return copied; +} + +/* ic starts out kzalloc()ed */ +void rds_iwarp_recv_init_ack(struct rds_iwarp_connection *ic) +{ + struct ib_send_wr *wr = &ic->i_ack_wr; + struct ib_sge *sge = &ic->i_ack_sge; + + sge->addr = ic->i_ack_dma; + sge->length = sizeof(struct rds_header); + sge->lkey = ic->i_mr->lkey; + + wr->sg_list = sge; + wr->num_sge = 1; + wr->opcode = IB_WR_SEND; + wr->wr_id = RDS_IWARP_ACK_WR_ID; + wr->send_flags = IB_SEND_SIGNALED | IB_SEND_SOLICITED; +} + +/* + * You'd think that with reliable IB connections you wouldn't need to ack + * messages that have been received. The problem is that IB hardware generates + * an ack message before it has DMAed the message into memory. This creates a + * potential message loss if the HCA is disabled for any reason between when it + * sends the ack and before the message is DMAed and processed. This is only a + * potential issue if another HCA is available for fail-over. + * + * When the remote host receives our ack they'll free the sent message from + * their send queue. To decrease the latency of this we always send an ack + * immediately after we've received messages. + * + * For simplicity, we only have one ack in flight at a time. This puts + * pressure on senders to have deep enough send queues to absorb the latency of + * a single ack frame being in flight. This might not be good enough. + * + * This is implemented by have a long-lived send_wr and sge which point to a + * statically allocated ack frame. This ack wr does not fall under the ring + * accounting that the tx and rx wrs do. The QP attribute specifically makes + * room for it beyond the ring size. Send completion notices its special + * wr_id and avoids working with the ring in that case. + */ +#ifndef KERNEL_HAS_ATOMIC64 +static void rds_iwarp_set_ack(struct rds_iwarp_connection *ic, u64 seq, + int ack_required) +{ + unsigned long flags; + + spin_lock_irqsave(&ic->i_ack_lock, flags); + if (ack_required) + set_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); + if (seq > ic->i_ack_next) + ic->i_ack_next = seq; + spin_unlock_irqrestore(&ic->i_ack_lock, flags); +} + +static u64 rds_iwarp_get_ack(struct rds_iwarp_connection *ic) +{ + unsigned long flags; + u64 seq; + + clear_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); + + spin_lock_irqsave(&ic->i_ack_lock, flags); + seq = ic->i_ack_next; + spin_unlock_irqrestore(&ic->i_ack_lock, flags); + + return seq; +} +#else +static void rds_iwarp_set_ack(struct rds_iwarp_connection *ic, u64 seq, + int ack_required) +{ + atomic64_set(&ic->i_ack_next, seq); + if (ack_required) { + smp_mb__before_clear_bit(); + set_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); + } +} + +static u64 rds_iwarp_get_ack(struct rds_iwarp_connection *ic) +{ + clear_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); + smp_mb__after_clear_bit(); + + return atomic64_read(&ic->i_ack_next); +} +#endif + + +static void rds_iwarp_send_ack(struct rds_iwarp_connection *ic, unsigned int adv_credits) +{ + struct rds_header *hdr = ic->i_ack; + struct ib_send_wr *failed_wr; + u64 seq; + int ret; + + seq = rds_iwarp_get_ack(ic); + + rdsdebug("send_ack: ic %p ack %llu\n", ic, (unsigned long long) seq); + rds_message_populate_header(hdr, 0, 0, 0); + hdr->h_ack = cpu_to_be64(seq); + hdr->h_credit = adv_credits; + rds_message_make_checksum(hdr); + ic->i_ack_queued = jiffies; + + ret = ib_post_send(ic->i_cm_id->qp, &ic->i_ack_wr, &failed_wr); + if (unlikely(ret)) { + /* Failed to send. Release the WR, and + * force another ACK. + */ + clear_bit(IWARP_ACK_IN_FLIGHT, &ic->i_ack_flags); + set_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); + + rds_iwarp_stats_inc(s_iwarp_ack_send_failure); + /* Need to finesse this later. */ + BUG(); + } else + rds_iwarp_stats_inc(s_iwarp_ack_sent); +} + +/* + * There are 3 ways of getting acknowledgements to the peer: + * 1. We call rds_iwarp_attempt_ack from the recv completion handler + * to send an ACK-only frame. + * However, there can be only one such frame in the send queue + * at any time, so we may have to postpone it. + * 2. When another (data) packet is transmitted while there's + * an ACK in the queue, we piggyback the ACK sequence number + * on the data packet. + * 3. If the ACK WR is done sending, we get called from the + * send queue completion handler, and check whether there's + * another ACK pending (postponed because the WR was on the + * queue). If so, we transmit it. + * + * We maintain 2 variables: + * - i_ack_flags, which keeps track of whether the ACK WR + * is currently in the send queue or not (IWARP_ACK_IN_FLIGHT) + * - i_ack_next, which is the last sequence number we received + * + * Potentially, send queue and receive queue handlers can run concurrently. + * It would be nice to not have to use a spinlock to synchronize things, + * but the one problem that rules this out is that 64bit updates are + * not atomic on all platforms. Things would be a lot simpler if + * we had atomic64 or maybe cmpxchg64 everywhere. + * + * Reconnecting complicates this picture just slightly. When we + * reconnect, we may be seeing duplicate packets. The peer + * is retransmitting them, because it hasn't seen an ACK for + * them. It is important that we ACK these. + * + * ACK mitigation adds a header flag "ACK_REQUIRED"; any packet with + * this flag set *MUST* be acknowledged immediately. + */ + +/* + * When we get here, we're called from the recv queue handler. + * Check whether we ought to transmit an ACK. + */ +void rds_iwarp_attempt_ack(struct rds_iwarp_connection *ic) +{ + unsigned int adv_credits; + + if (!test_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags)) + return; + + if (test_and_set_bit(IWARP_ACK_IN_FLIGHT, &ic->i_ack_flags)) { + rds_iwarp_stats_inc(s_iwarp_ack_send_delayed); + return; + } + + /* Can we get a send credit? */ + if (!rds_iwarp_send_grab_credits(ic, 1, &adv_credits)) { + rds_iwarp_stats_inc(s_iwarp_tx_throttle); + clear_bit(IWARP_ACK_IN_FLIGHT, &ic->i_ack_flags); + return; + } + + clear_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); + rds_iwarp_send_ack(ic, adv_credits); +} + +/* + * We get here from the send completion handler, when the + * adapter tells us the ACK frame was sent. + */ +void rds_iwarp_ack_send_complete(struct rds_iwarp_connection *ic) +{ + clear_bit(IWARP_ACK_IN_FLIGHT, &ic->i_ack_flags); + rds_iwarp_attempt_ack(ic); +} + +/* + * This is called by the regular xmit code when it wants to piggyback + * an ACK on an outgoing frame. + */ +u64 rds_iwarp_piggyb_ack(struct rds_iwarp_connection *ic) +{ + if (test_and_clear_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags)) + rds_iwarp_stats_inc(s_iwarp_ack_send_piggybacked); + return rds_iwarp_get_ack(ic); +} + +/* + * Work is posted as a RDS_FRAG_SIZE payload and then a header. This is + * done so that we can send fragments without headers and keep the fragments + * large and aligned. The sender doesn't pad their fragments so the header + * will spill into the posted regions just after the fragment. + * + * XXX If we were to flip r_page into userspace or the page cache then we'd + * have to zero the header and possibly the rest of the page. + */ +static int rds_iwarp_copy_header(struct rds_connection *conn, + struct rds_header *hdr, + struct rds_iwarp_recv_work *recv, u32 start) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + void *dst = hdr; + void *addr; + u32 len = 0; + + /* get the start of the header from the tail of the fragment */ + if (start < RDS_FRAG_SIZE) { + len = min_t(u32, RDS_FRAG_SIZE - start, + sizeof(struct rds_header)); + addr = kmap_atomic(recv->r_frag->f_page, KM_SOFTIRQ0); + memcpy(dst, + addr + recv->r_frag->f_offset + start, + len); + kunmap_atomic(addr, KM_SOFTIRQ0); + dst += len; + } + + /* and the rest that might have spilled into the posted header space */ + if (len < sizeof(struct rds_header)) { + memcpy(dst, + &ic->i_recv_hdrs[recv - ic->i_recvs], + sizeof(struct rds_header) - len); + } + + return 1; +} + +/* + * It's kind of lame that we're copying from the posted receive pages into + * long-lived bitmaps. We could have posted the bitmaps and rdma written into + * them. But receiving new congestion bitmaps should be a *rare* event, so + * hopefully we won't need to invest that complexity in making it more + * efficient. By copying we can share a simpler core with TCP which has to + * copy. + */ +static void rds_iwarp_cong_recv(struct rds_connection *conn, + struct rds_iwarp_incoming *ibinc) +{ + struct rds_cong_map *map; + unsigned int map_off; + unsigned int map_page; + struct rds_page_frag *frag; + unsigned long frag_off; + unsigned long to_copy; + unsigned long copied; + uint64_t uncongested = 0; + void *addr; + + /* catch completely corrupt packets */ + if (be32_to_cpu(ibinc->ii_inc.i_hdr.h_len) != RDS_CONG_MAP_BYTES) + return; + + map = conn->c_fcong; + map_page = 0; + map_off = 0; + + frag = list_entry(ibinc->ii_frags.next, struct rds_page_frag, f_item); + frag_off = 0; + + copied = 0; + + while (copied < RDS_CONG_MAP_BYTES) { + uint64_t *src, *dst; + unsigned int k; + + to_copy = min(RDS_FRAG_SIZE - frag_off, PAGE_SIZE - map_off); + BUG_ON(to_copy & 7); /* Must be 64bit aligned. */ + + addr = kmap_atomic(frag->f_page, KM_SOFTIRQ0); + + src = addr + frag_off; + dst = (void *)map->m_page_addrs[map_page] + map_off; + for (k = 0; k < to_copy; k += 8) { + /* Record ports that became uncongested, ie + * bits that changed from 0 to 1. */ + uncongested |= ~(*src) & *dst; + *dst++ = *src++; + } + kunmap_atomic(addr, KM_SOFTIRQ0); + + copied += to_copy; + + map_off += to_copy; + if (map_off == PAGE_SIZE) { + map_off = 0; + map_page++; + } + + frag_off += to_copy; + if (frag_off == RDS_FRAG_SIZE) { + frag = list_entry(frag->f_item.next, + struct rds_page_frag, f_item); + frag_off = 0; + } + } + + /* the congestion map is in little endian order */ + uncongested = le64_to_cpu(uncongested); + + rds_cong_map_updated(map, uncongested); +} + +/* + * Rings are posted with all the allocations they'll need to queue the + * incoming message to the receiving socket so this can't fail. + * All fragments start with a header, so we can make sure we're not receiving + * garbage, and we can tell a small 8 byte fragment from an ACK frame. + */ +struct rds_iwarp_ack_state { + u64 ack_next; + u64 ack_recv; + int ack_required : 1, + ack_next_valid : 1, + ack_recv_valid : 1; +}; +static void rds_iwarp_process_recv(struct rds_connection *conn, + struct rds_iwarp_recv_work *recv, u32 byte_len, + struct rds_iwarp_ack_state *state) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct rds_iwarp_incoming *ibinc = ic->i_iwarpinc; + struct rds_header hdr_buf, *ihdr, *hdr; + + /* XXX shut down the connection if port 0,0 are seen? */ + + rdsdebug("ic %p ibinc %p recv %p byte len %u\n", ic, ibinc, recv, + byte_len); + + if (byte_len < sizeof(struct rds_header)) { + rds_iwarp_conn_error(conn, "incoming message " + "from %u.%u.%u.%u didn't inclue a " + "header, disconnecting and " + "reconnecting\n", + NIPQUAD(conn->c_faddr)); + return; + } + byte_len -= sizeof(struct rds_header); + + if (ic->i_hdr_idx == 0) + ihdr = &ic->i_recv_hdrs[recv - ic->i_recvs]; + else if (rds_iwarp_copy_header(conn, &hdr_buf, recv, byte_len)) + ihdr = &hdr_buf; + else + return; + + /* Validate the checksum. */ + if (!rds_message_verify_checksum(ihdr)) { + rds_iwarp_conn_error(conn, "incoming message " + "from %u.%u.%u.%u has corrupted header - " + "forcing a reconnect\n", + NIPQUAD(conn->c_faddr)); + rds_stats_inc(s_recv_drop_bad_checksum); + return; + } + + /* Process the ACK sequence which comes with every packet */ + state->ack_recv = be64_to_cpu(ihdr->h_ack); + state->ack_recv_valid = 1; + + /* Process the credits update if there was one */ + if (ihdr->h_credit) + rds_iwarp_send_add_credits(conn, ihdr->h_credit); + + if (ihdr->h_sport == 0 && ihdr->h_dport == 0 && byte_len == 0) { + /* This is an ACK-only packet. The fact that it gets + * special treatment here is that historically, ACKs + * were rather special beasts. + */ + rds_iwarp_stats_inc(s_iwarp_ack_received); + + /* + * Usually the frags make their way on to incs and are then freed as + * the inc is freed. We don't go that route, so we have to drop the + * page ref ourselves. We can't just leave the page on the recv + * because that confuses the dma mapping of pages and each recv's use + * of a partial page. We can leave the frag, though, it will be + * reused. + * + * FIXME: Fold this into the code path below. + */ + rds_iwarp_frag_drop_page(recv->r_frag); + return; + } + + /* + * If we don't already have an inc on the connection then this + * fragment has a header and starts a message.. copy its header + * into the inc and save the inc so we can hang upcoming fragments + * off its list. + */ + if (ibinc == NULL) { + ibinc = recv->r_iwarpinc; + recv->r_iwarpinc = NULL; + ic->i_iwarpinc = ibinc; + + hdr = &ibinc->ii_inc.i_hdr; + memcpy(hdr, ihdr, sizeof(*hdr)); + ic->i_recv_data_rem = be32_to_cpu(hdr->h_len); + + rdsdebug("ic %p ibinc %p rem %u flag 0x%x\n", ic, ibinc, + ic->i_recv_data_rem, hdr->h_flags); + } else { + hdr = &ibinc->ii_inc.i_hdr; + /* We can't just use memcmp here; fragments of a + * single message may carry different ACKs */ + if (hdr->h_sequence != ihdr->h_sequence + || hdr->h_len != ihdr->h_len + || hdr->h_sport != ihdr->h_sport + || hdr->h_dport != ihdr->h_dport) { + rds_iwarp_conn_error(conn, + "fragment header mismatch; forcing reconnect\n"); + return; + } + } + + list_add_tail(&recv->r_frag->f_item, &ibinc->ii_frags); + recv->r_frag = NULL; + + if (ic->i_recv_data_rem > RDS_FRAG_SIZE) + ic->i_recv_data_rem -= RDS_FRAG_SIZE; + else { + ic->i_recv_data_rem = 0; + ic->i_iwarpinc = NULL; + + if (ibinc->ii_inc.i_hdr.h_flags == RDS_FLAG_CONG_BITMAP) + rds_iwarp_cong_recv(conn, ibinc); + else + rds_recv_incoming(conn, conn->c_faddr, conn->c_laddr, + &ibinc->ii_inc, GFP_ATOMIC, + KM_SOFTIRQ0); + + /* Evaluate the ACK_REQUIRED flag *after* we received + * the complete frame, and after bumping the next_rx + * sequence. */ + if (hdr->h_flags & RDS_FLAG_ACK_REQUIRED) { + rds_stats_inc(s_recv_ack_required); + state->ack_required = 1; + } + + state->ack_next = be64_to_cpu(hdr->h_sequence); + state->ack_next_valid = 1; + rds_inc_put(&ibinc->ii_inc); + } +} + +/* + * Plucking the oldest entry from the ring can be done concurrently with + * the thread refilling the ring. Each ring operation is protected by + * spinlocks and the transient state of refilling doesn't change the + * recording of which entry is oldest. + * + * This relies on IB only calling one cq comp_handler for each cq so that + * there will only be one caller of rds_recv_incoming() per RDS connection. + */ +void rds_iwarp_recv_cq_comp_handler(struct ib_cq *cq, void *context) +{ + struct rds_connection *conn = context; + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct ib_wc wc; + struct rds_iwarp_ack_state state = { 0, }; + struct rds_iwarp_recv_work *recv; + int ret = 0; + + rdsdebug("conn %p cq %p\n", conn, cq); + + rds_iwarp_stats_inc(s_iwarp_rx_cq_call); + + ib_req_notify_cq(cq, IB_CQ_SOLICITED); + + while (ib_poll_cq(cq, 1, &wc) > 0) { + rdsdebug("wc wr_id 0x%llx status %u byte_len %u imm_data %u\n", + (unsigned long long)wc.wr_id, wc.status, wc.byte_len, + be32_to_cpu(wc.ex.imm_data)); + rds_iwarp_stats_inc(s_iwarp_rx_cq_event); + + recv = &ic->i_recvs[rds_iwarp_ring_oldest(&ic->i_recv_ring)]; +#if 0 + if (recv->r_sge[1].addr == 0) + printk("recv CQ ring: alloc ptr=%u/ctr=%u free ptr=%u/ctr=%u\n", + ic->i_recv_ring.w_alloc_ptr, + ic->i_recv_ring.w_alloc_ctr, + ic->i_recv_ring.w_free_ptr, + atomic_read(&ic->i_recv_ring.w_free_ctr)); +#endif + rds_iwarp_recv_unmap_page(ic, recv); + + if (rds_conn_up(conn)) { + /* We expect errors as the qp is drained during shutdown */ + if (wc.status == IB_WC_SUCCESS) { + rds_iwarp_process_recv(conn, recv, wc.byte_len, &state); + } else { + rds_iwarp_conn_error(conn, "recv completion on " + "%u.%u.%u.%u had status %u, disconnecting and " + "reconnecting\n", NIPQUAD(conn->c_faddr), + wc.status); + } + } + + rds_iwarp_ring_free(&ic->i_recv_ring, 1); + } + + if (state.ack_next_valid) + rds_iwarp_set_ack(ic, state.ack_next, state.ack_required); + if (state.ack_recv_valid && state.ack_recv > ic->i_ack_recv) { + rds_send_drop_acked(conn, state.ack_recv, NULL); + ic->i_ack_recv = state.ack_recv; + } + if (rds_conn_up(conn)) + rds_iwarp_attempt_ack(ic); + + /* + * XXX atomic is bad as it drains reserve pools, we should really + * do some non-blocking alloc that doesn't touch the pools but + * will fail. Then leave it to the thread to get to reclaim + * and alloc. + */ + + /* + * If we fail to refill we assume it's a allocation failure + * from our use of GFP_ATOMIC and we want the thread to try again + * immediately. Similarly, if the thread is already trying to + * refill we want it to try again immediately as it may have missed + * the ring entry we just completed before it released the + * i_recv_mutex. + */ + /* If we ever end up with a really empty receive ring, we're + * in deep trouble, as the sender will definitely see RNR + * timeouts. */ + if (rds_iwarp_ring_empty(&ic->i_recv_ring)) + rds_iwarp_stats_inc(s_iwarp_rx_ring_empty); + + if (mutex_trylock(&ic->i_recv_mutex)) { + if (rds_iwarp_recv_refill(conn, GFP_ATOMIC, + GFP_ATOMIC | __GFP_HIGHMEM, 0)) + ret = -EAGAIN; + else + rds_iwarp_stats_inc(s_iwarp_rx_refill_from_cq); + mutex_unlock(&ic->i_recv_mutex); + } else + ret = -EAGAIN; + + if (ret) + queue_delayed_work(rds_wq, &conn->c_recv_w, 0); +} + +int rds_iwarp_recv(struct rds_connection *conn) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + int ret = 0; + + rdsdebug("conn %p\n", conn); + + /* + * If we get a temporary posting failure in this context then + * we're really low and we want the caller to back off for a bit. + */ + mutex_lock(&ic->i_recv_mutex); + if (rds_iwarp_recv_refill(conn, GFP_KERNEL, GFP_HIGHUSER, 0)) + ret = -ENOMEM; + else + rds_iwarp_stats_inc(s_iwarp_rx_refill_from_thread); + mutex_unlock(&ic->i_recv_mutex); + + return ret; +} + +int __init rds_iwarp_recv_init(void) +{ + struct sysinfo si; + int ret = -ENOMEM; + + /* Default to 30% of all available RAM for recv memory */ + si_meminfo(&si); + rds_iwarp_sysctl_max_recv_allocation = si.totalram / 3 * PAGE_SIZE / RDS_FRAG_SIZE; + + rds_iwarp_incoming_slab = kmem_cache_create("rds_iwarp_incoming", + sizeof(struct rds_iwarp_incoming), + 0, 0, NULL); + if (rds_iwarp_incoming_slab == NULL) + goto out; + + rds_iwarp_frag_slab = kmem_cache_create("rds_iwarp_frag", + sizeof(struct rds_page_frag), + 0, 0, NULL); + if (rds_iwarp_frag_slab == NULL) + kmem_cache_destroy(rds_iwarp_incoming_slab); + else + ret = 0; +out: + return ret; +} + +void rds_iwarp_recv_exit(void) +{ + kmem_cache_destroy(rds_iwarp_incoming_slab); + kmem_cache_destroy(rds_iwarp_frag_slab); +} diff --git a/net/rds/iwarp_ring.c b/net/rds/iwarp_ring.c new file mode 100644 index 0000000..64361e9 --- /dev/null +++ b/net/rds/iwarp_ring.c @@ -0,0 +1,156 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include + +#define DEBUG +#include "rds.h" +#include "iwarp.h" + +/* + * Locking for IB rings. + * We assume that allocation is always protected by a mutex + * in the caller (this is a valid assumption for the current + * implementation). + * + * Freeing always happens in an interrupt, and hence only + * races with allocations, but not with other free()s. + * + * The interaction between allocation and freeing is that + * the alloc code has to determine the number of free entries. + * To this end, we maintain two counters; an allocation counter + * and a free counter. Both are allowed to run freely, and wrap + * around. + * The number of used entries is always (alloc_ctr - free_ctr) % NR. + * + * The current implementation makes free_ctr atomic. When the + * caller finds an allocation fails, it should set an "alloc fail" + * bit and retry the allocation. The "alloc fail" bit essentially tells + * the CQ completion handlers to wake it up after freeing some + * more entries. + */ + +/* + * This only happens on shutdown. + */ +DECLARE_WAIT_QUEUE_HEAD(rds_iwarp_ring_empty_wait); + +void rds_iwarp_ring_init(struct rds_iwarp_work_ring *ring, u32 nr) +{ + printk(KERN_DEBUG "rds_iwarp_ring_init(%u)\n", nr); + memset(ring, 0, sizeof(*ring)); + ring->w_nr = nr; +} + +static inline u32 __rds_iwarp_ring_used(struct rds_iwarp_work_ring *ring) +{ + u32 diff; + + /* This assumes that atomic_t has at least as many bits as u32 */ + diff = ring->w_alloc_ctr - (u32) atomic_read(&ring->w_free_ctr); + BUG_ON(diff > ring->w_nr); + + return diff; +} + +static int __rds_iwarp_ring_empty(struct rds_iwarp_work_ring *ring) +{ + return __rds_iwarp_ring_used(ring) == 0; +} + +u32 rds_iwarp_ring_alloc(struct rds_iwarp_work_ring *ring, u32 val, u32 *pos) +{ + u32 ret = 0, avail; + + avail = ring->w_nr - __rds_iwarp_ring_used(ring); + + rdsdebug("ring %p val %u next %u free %u\n", ring, val, + ring->w_alloc_ptr, avail); + + if (val && avail) { + ret = min(val, avail); + *pos = ring->w_alloc_ptr; + + ring->w_alloc_ptr = (ring->w_alloc_ptr + ret) % ring->w_nr; + ring->w_alloc_ctr += ret; + } + + return ret; +} + +void rds_iwarp_ring_free(struct rds_iwarp_work_ring *ring, u32 val) +{ + ring->w_free_ptr = (ring->w_free_ptr + val) % ring->w_nr; + atomic_add(val, &ring->w_free_ctr); + + if (__rds_iwarp_ring_empty(ring) && + waitqueue_active(&rds_iwarp_ring_empty_wait)) + wake_up(&rds_iwarp_ring_empty_wait); +} + +void rds_iwarp_ring_unalloc(struct rds_iwarp_work_ring *ring, u32 val) +{ + ring->w_alloc_ptr = (ring->w_alloc_ptr - val) % ring->w_nr; + ring->w_alloc_ctr -= val; +} + +int rds_iwarp_ring_empty(struct rds_iwarp_work_ring *ring) +{ + return __rds_iwarp_ring_empty(ring); +} + +/* + * returns the oldest alloced ring entry. This will be the next one + * freed. This can't be called if there are none allocated. + */ +u32 rds_iwarp_ring_oldest(struct rds_iwarp_work_ring *ring) +{ + return ring->w_free_ptr; +} + +/* + * returns the number of completed work requests. + */ + +u32 rds_iwarp_ring_completed(struct rds_iwarp_work_ring *ring, u32 wr_id, u32 oldest) +{ + u32 ret; + + if (oldest <= (unsigned long long)wr_id) + ret = (unsigned long long)wr_id - oldest + 1; + else + ret = ring->w_nr - oldest + (unsigned long long)wr_id + 1; + + rdsdebug("ring %p ret %u wr_id %u oldest %u\n", ring, ret, + wr_id, oldest); + return ret; +} diff --git a/net/rds/iwarp_send.c b/net/rds/iwarp_send.c new file mode 100644 index 0000000..434adf8 --- /dev/null +++ b/net/rds/iwarp_send.c @@ -0,0 +1,826 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include +#include +#include +#include + +#include "rds.h" +#include "rdma.h" +#include "iwarp.h" + +void rds_iwarp_send_unmap_rm(struct rds_iwarp_connection *ic, + struct rds_iwarp_send_work *send, + int success) +{ + struct rds_message *rm = send->s_rm; + + rdsdebug("ic %p send %p rm %p\n", ic, send, rm); + + ib_dma_unmap_sg(ic->i_cm_id->device, + rm->m_sg, rm->m_nents, + DMA_TO_DEVICE); + + /* raise rdma completion hwm */ + if (rm->m_rdma_op && success) { + struct rds_rdma_op *op = rm->m_rdma_op; + + /* If this was a RDMA READ, make sure the CPU sees all + * the updates. */ + if (!op->r_write) + ib_dma_sync_sg_for_cpu(ic->i_cm_id->device, + op->r_sg, op->r_count, + DMA_FROM_DEVICE); + + /* If the user asked for a completion notification on this + * message, we can implement three different semantics: + * 1. Notify when we received the ACK on the RDS message + * that was queued with the RDMA. This provides reliable + * notification of RDMA status at the expense of a one-way + * packet delay. + * 2. Notify when the IB stack gives us the completion event for + * the RDMA operation. + * 3. Notify when the IB stack gives us the completion event for + * the accompanying RDS messages. + * Here, we implement approach #3. To implement approach #2, + * call rds_rdma_send_complete from the cq_handler. To implement #1, + * don't call rds_rdma_send_complete at all, and fall back to the notify + * handling in the ACK processing code. + */ + rds_rdma_send_complete(rm); + + if (rm->m_rdma_op->r_write) + rds_stats_add(s_send_rdma_bytes, rm->m_rdma_op->r_bytes); + else + rds_stats_add(s_recv_rdma_bytes, rm->m_rdma_op->r_bytes); + } + + /* If anyone waited for this message to get flushed out, wake + * them up now */ + rds_message_unmapped(rm); + + rds_message_put(rm); + send->s_rm = NULL; +} + +void rds_iwarp_send_init_ring(struct rds_iwarp_connection *ic) +{ + struct rds_iwarp_send_work *send; + u32 i; + + for(i = 0, send = ic->i_sends; i < ic->i_send_ring.w_nr; i++, send++) { + struct ib_sge *sge; + + send->s_rm = NULL; + send->s_op = NULL; + + send->s_wr.wr_id = i; + send->s_wr.sg_list = send->s_sge; + send->s_wr.num_sge = 1; + send->s_wr.opcode = IB_WR_SEND; + send->s_wr.send_flags = 0; + //send->s_wr.imm_data = 0; + + sge = rds_iwarp_data_sge(ic, send->s_sge); + sge->lkey = ic->i_mr->lkey; + + sge = rds_iwarp_header_sge(ic, send->s_sge); + sge->addr = ic->i_send_hdrs_dma + (i * sizeof(struct rds_header)); + sge->length = sizeof(struct rds_header); + sge->lkey = ic->i_mr->lkey; + } +} + +void rds_iwarp_send_clear_ring(struct rds_iwarp_connection *ic) +{ + struct rds_iwarp_send_work *send; + u32 i; + + for(i = 0, send = ic->i_sends; i < ic->i_send_ring.w_nr; i++, send++) { + if (send->s_rm) + rds_iwarp_send_unmap_rm(ic, send, 0); + if (send->s_op) + ib_dma_unmap_sg(ic->i_cm_id->device, + send->s_op->r_sg, send->s_op->r_nents, + send->s_op->r_write? DMA_TO_DEVICE : DMA_FROM_DEVICE); + } +} + +/* + * The _oldest/_free ring operations here race cleanly with the alloc/unalloc + * operations performed in the send path. As the sender allocs and potentially + * unallocs the next free entry in the ring it doesn't alter which is + * the next to be freed, which is what this is concerned with. + */ +void rds_iwarp_send_cq_comp_handler(struct ib_cq *cq, void *context) +{ + struct rds_connection *conn = context; + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct ib_wc wc; + struct rds_iwarp_send_work *send; + u32 completed; + u32 oldest; + u32 i = 0; + int ret; + + rdsdebug("cq %p conn %p\n", cq, conn); + rds_iwarp_stats_inc(s_iwarp_tx_cq_call); + ret = ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); + if (ret) { + rdsdebug("ib_req_notify_cq send failed: %d\n", ret); + } + + while (ib_poll_cq(cq, 1, &wc) > 0 ) { +/* + rdsdebug("wc wr_id 0x%llx status %u byte_len %u imm_data %u\n", + (unsigned long long)wc.wr_id, wc.status, wc.byte_len, + be32_to_cpu(wc.imm_data)); +*/ rds_iwarp_stats_inc(s_iwarp_tx_cq_event); + + if (wc.wr_id == RDS_IWARP_ACK_WR_ID) { + if (ic->i_ack_queued + HZ/2 < jiffies) + rds_iwarp_stats_inc(s_iwarp_tx_stalled); + rds_iwarp_ack_send_complete(ic); + continue; + } + + oldest = rds_iwarp_ring_oldest(&ic->i_send_ring); + + completed = rds_iwarp_ring_completed(&ic->i_send_ring, wc.wr_id, oldest); + + for (i = 0; i < completed; i++) { + send = &ic->i_sends[oldest]; + + /* In the error case, wc.opcode sometimes contains garbage */ + switch (send->s_wr.opcode) { + case IB_WR_SEND: + if (send->s_rm) + rds_iwarp_send_unmap_rm(ic, send, 1); + break; + case IB_WR_RDMA_WRITE: + if (send->s_op) + ib_dma_unmap_sg(ic->i_cm_id->device, + send->s_op->r_sg, send->s_op->r_nents, + DMA_TO_DEVICE); + break; + case IB_WR_RDMA_READ: + if (send->s_op) + ib_dma_unmap_sg(ic->i_cm_id->device, + send->s_op->r_sg, send->s_op->r_nents, + DMA_FROM_DEVICE); + break; + default: + if (printk_ratelimit()) + printk(KERN_NOTICE + "RDS/IWARP: %s: unexpected opcode 0x%x in WR!\n", + __FUNCTION__, send->s_wr.opcode); + break; + } + + send->s_wr.opcode = 0xdead; + send->s_wr.num_sge = 1; + if (send->s_queued + HZ/2 < jiffies) + rds_iwarp_stats_inc(s_iwarp_tx_stalled); + + oldest = (oldest + 1) % ic->i_send_ring.w_nr; + } + + if (unlikely(wc.status != IB_WC_SUCCESS && send->s_op && send->s_op->r_notifier)) { + switch (wc.status) { + default: + send->s_op->r_notifier->n_status = RDS_RDMA_OTHER_ERROR; + break; + case IB_WC_REM_ACCESS_ERR: + send->s_op->r_notifier->n_status = RDS_RDMA_REMOTE_ERROR; + break; + case IB_WC_WR_FLUSH_ERR: + /* flushed out; not an error */ + break; + } + } + + rds_iwarp_ring_free(&ic->i_send_ring, completed); + + if (test_and_clear_bit(RDS_LL_SEND_FULL, &conn->c_flags) + || test_bit(0, &conn->c_map_queued)) + queue_delayed_work(rds_wq, &conn->c_send_w, 0); + + /* We expect errors as the qp is drained during shutdown */ + if (wc.status != IB_WC_SUCCESS && rds_conn_up(conn)) { + rds_iwarp_conn_error(conn, + "send completion on %u.%u.%u.%u " + "had status %u, disconnecting and reconnecting\n", + NIPQUAD(conn->c_faddr), wc.status); + } + } +} + +/* + * This is the main function for allocating credits when sending + * messages. + * + * Conceptually, we have two counters: + * - send credits: this tells us how many WRs we're allowed + * to submit without overruning the reciever's queue. For + * each SEND WR we post, we decrement this by one. + * + * - posted credits: this tells us how many WRs we recently + * posted to the receive queue. This value is transferred + * to the peer as a "credit update" in a RDS header field. + * Every time we transmit credits to the peer, we subtract + * the amount of transferred credits from this counter. + * + * It is essential that we avoid situations where both sides have + * exhausted their send credits, and are unable to send new credits + * to the peer. We achieve this by requiring that we send at least + * one credit update to the peer before exhausting our credits. + * When new credits arrive, we subtract one credit that is withheld + * until we've posted new buffers and are ready to transmit these + * credits (see rds_iwarp_send_add_credits below). + * + * The RDS send code is essentially single-threaded; rds_send_xmit + * grabs c_send_sem to ensure exclusive access to the send ring. + * However, the ACK sending code is independent and can race with + * message SENDs. + * + * In the send path, we need to update the counters for send credits + * and the counter of posted buffers atomically - when we use the + * last available credit, we cannot allow another thread to race us + * and grab the posted credits counter. Hence, we have to use a + * spinlock to protect the credit counter, or use atomics. + * + * Spinlocks shared between the send and the receive path are bad, + * because they create unnecessary delays. An early implementation + * using a spinlock showed a 5% degradation in throughput at some + * loads. + * + * This implementation avoids spinlocks completely, putting both + * counters into a single atomic, and updating that atomic using + * atomic_add (in the receive path, when receiving fresh credits), + * and using atomic_cmpxchg when updating the two counters. + */ +int rds_iwarp_send_grab_credits(struct rds_iwarp_connection *ic, + u32 wanted, u32 *adv_credits) +{ + unsigned int avail, posted, got = 0, advertise; + long oldval, newval; + + *adv_credits = 0; + if (!ic->i_flowctl) + return wanted; + +try_again: + advertise = 0; + oldval = newval = atomic_read(&ic->i_credits); + posted = IWARP_GET_POST_CREDITS(oldval); + avail = IWARP_GET_SEND_CREDITS(oldval); + + rdsdebug("rds_iwarp_send_grab_credits(%u): credits=%u posted=%u\n", + wanted, avail, posted); + + /* The last credit must be used to send a credit updated. */ + if (avail && !posted) + avail--; + + if (avail < wanted) { + struct rds_connection *conn = ic->i_cm_id->context; + + /* Oops, there aren't that many credits left! */ + set_bit(RDS_LL_SEND_FULL, &conn->c_flags); + got = avail; + } else { + /* Sometimes you get what you want, lalala. */ + got = wanted; + } + newval -= IWARP_SET_SEND_CREDITS(got); + + if (got && posted) { + advertise = min_t(unsigned int, posted, RDS_MAX_ADV_CREDIT); + newval -= IWARP_SET_POST_CREDITS(advertise); + } + + /* Finally bill everything */ + if (atomic_cmpxchg(&ic->i_credits, oldval, newval) != oldval) + goto try_again; + + *adv_credits = advertise; + return got; +} + +void rds_iwarp_send_add_credits(struct rds_connection *conn, unsigned int credits) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + + if (credits == 0) + return; + + rdsdebug("rds_iwarp_send_add_credits(%u): current=%u%s\n", + credits, + IWARP_GET_SEND_CREDITS(atomic_read(&ic->i_credits)), + test_bit(RDS_LL_SEND_FULL, &conn->c_flags)? ", ll_send_full" : ""); + + atomic_add(IWARP_SET_SEND_CREDITS(credits), &ic->i_credits); + if (test_and_clear_bit(RDS_LL_SEND_FULL, &conn->c_flags)) + queue_delayed_work(rds_wq, &conn->c_send_w, 0); + + WARN_ON(IWARP_GET_SEND_CREDITS(credits) >= 16384); + + rds_iwarp_stats_inc(s_iwarp_rx_credit_updates); +} + +void rds_iwarp_advertise_credits(struct rds_connection *conn, unsigned int posted) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + + if (posted == 0) + return; + + atomic_add(IWARP_SET_POST_CREDITS(posted), &ic->i_credits); + + /* Decide whether to send an update to the peer now. + * If we would send a credit update for every single buffer we + * post, we would end up with an ACK storm (ACK arrives, + * consumes buffer, we refill the ring, send ACK to remote + * advertising the newly posted buffer... ad inf) + * + * Performance pretty much depends on how often we send + * credit updates - too frequent updates mean lots of ACKs. + * Too infrequent updates, and the peer will run out of + * credits and has to throttle. + * For the time being, 16 seems to be a good compromise. + */ + if (IWARP_GET_POST_CREDITS(atomic_read(&ic->i_credits)) >= 16) + set_bit(IWARP_ACK_REQUESTED, &ic->i_ack_flags); +} + +static inline void +rds_iwarp_xmit_populate_wr(struct rds_iwarp_connection *ic, + struct rds_iwarp_send_work *send, unsigned int pos, + unsigned long buffer, unsigned int length, + int send_flags) +{ + struct ib_sge *sge; + + WARN_ON(pos != send - ic->i_sends); + + send->s_wr.send_flags = send_flags; + send->s_wr.opcode = IB_WR_SEND; + send->s_wr.num_sge = 2; + send->s_wr.next = NULL; + send->s_queued = jiffies; + send->s_op = NULL; + + if (length != 0) { + sge = rds_iwarp_data_sge(ic, send->s_sge); + sge->addr = buffer; + sge->length = length; + sge->lkey = ic->i_mr->lkey; + + sge = rds_iwarp_header_sge(ic, send->s_sge); + } else { + /* We're sending a packet with no payload. There is only + * one SGE */ + send->s_wr.num_sge = 1; + sge = &send->s_sge[0]; + } + + sge->addr = ic->i_send_hdrs_dma + (pos * sizeof(struct rds_header)); + sge->length = sizeof(struct rds_header); + sge->lkey = ic->i_mr->lkey; +} + +/* + * This can be called multiple times for a given message. The first time + * we see a message we map its scatterlist into the IB device so that + * we can provide that mapped address to the IB scatter gather entries + * in the IB work requests. We translate the scatterlist into a series + * of work requests that fragment the message. These work requests complete + * in order so we pass ownership of the message to the completion handler + * once we send the final fragment. + * + * The RDS core uses the c_send_sem to only enter this function once + * per connection. This makes sure that the tx ring alloc/unalloc pairs + * don't get out of sync and confuse the ring. + */ +int rds_iwarp_xmit(struct rds_connection *conn, struct rds_message *rm, + unsigned int hdr_off, unsigned int sg, unsigned int off) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct ib_device *dev = ic->i_cm_id->device; + struct rds_iwarp_send_work *send = NULL; + struct rds_iwarp_send_work *first; + struct rds_iwarp_send_work *prev; + struct ib_send_wr *failed_wr; + struct scatterlist *scat; + u32 pos; + u32 i; + u32 work_alloc; + u32 credit_alloc; + u32 adv_credits = 0; + int send_flags = 0; + int sent; + int ret; + + BUG_ON(off % RDS_FRAG_SIZE); + BUG_ON(hdr_off != 0 && hdr_off != sizeof(struct rds_header)); + + /* FIXME we may overallocate here */ + if (be32_to_cpu(rm->m_inc.i_hdr.h_len) == 0) + i = 1; + else + i = ceil(be32_to_cpu(rm->m_inc.i_hdr.h_len), RDS_FRAG_SIZE); + + work_alloc = rds_iwarp_ring_alloc(&ic->i_send_ring, i, &pos); + if (work_alloc == 0) { + set_bit(RDS_LL_SEND_FULL, &conn->c_flags); + rds_iwarp_stats_inc(s_iwarp_tx_ring_full); + ret = -ENOMEM; + goto out; + } + + credit_alloc = work_alloc; + if (ic->i_flowctl) { + credit_alloc = rds_iwarp_send_grab_credits(ic, work_alloc, &adv_credits); + if (credit_alloc < work_alloc) { + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc - credit_alloc); + work_alloc = credit_alloc; + } + if (work_alloc == 0) { + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc); + rds_iwarp_stats_inc(s_iwarp_tx_throttle); + ret = -ENOMEM; + goto out; + } + } + + /* map the message the first time we see it */ + if (ic->i_rm == NULL) { + /* + printk(KERN_NOTICE "rds_iwarp_xmit prep msg dport=%u flags=0x%x len=%d\n", + be16_to_cpu(rm->m_inc.i_hdr.h_dport), + rm->m_inc.i_hdr.h_flags, + be32_to_cpu(rm->m_inc.i_hdr.h_len)); + */ + if (rm->m_nents) { + rm->m_count = ib_dma_map_sg(dev, + rm->m_sg, rm->m_nents, DMA_TO_DEVICE); + rdsdebug("ic %p mapping rm %p: %d\n", ic, rm, rm->m_count); + if (rm->m_count == 0) { + rds_iwarp_stats_inc(s_iwarp_tx_sg_mapping_failure); + ret = -ENOMEM; /* XXX ? */ + goto out; + } + } else { + rm->m_count = 0; + } + + ic->i_unsignaled_wrs = rds_iwarp_sysctl_max_unsig_wrs; + ic->i_unsignaled_bytes = rds_iwarp_sysctl_max_unsig_bytes; + rds_message_addref(rm); + ic->i_rm = rm; + + /* Finalize the header */ + if (test_bit(RDS_MSG_ACK_REQUIRED, &rm->m_flags)) + rm->m_inc.i_hdr.h_flags |= RDS_FLAG_ACK_REQUIRED; + if (test_bit(RDS_MSG_RETRANSMITTED, &rm->m_flags)) + rm->m_inc.i_hdr.h_flags |= RDS_FLAG_RETRANSMITTED; + + /* If it has a RDMA op, tell the peer we did it. This is + * used by the peer to release use-once RDMA MRs. */ + if (rm->m_rdma_op) { + struct rds_ext_header_rdma ext_hdr; + + ext_hdr.h_rdma_rkey = cpu_to_be32(rm->m_rdma_op->r_key); + rds_message_add_extension(&rm->m_inc.i_hdr, + RDS_EXTHDR_RDMA, &ext_hdr, sizeof(ext_hdr)); + } + if (rm->m_rdma_cookie) { + rds_message_add_rdma_dest_extension(&rm->m_inc.i_hdr, + rds_rdma_cookie_key(rm->m_rdma_cookie), + rds_rdma_cookie_offset(rm->m_rdma_cookie)); + } + + /* Note - rds_iwarp_piggyb_ack clears the ACK_REQUIRED bit, so + * we should not do this unless we have a chance of at least + * sticking the header into the send ring. Which is why we + * should call rds_iwarp_ring_alloc first. */ + rm->m_inc.i_hdr.h_ack = cpu_to_be64(rds_iwarp_piggyb_ack(ic)); + rds_message_make_checksum(&rm->m_inc.i_hdr); + } else if (ic->i_rm != rm) + BUG(); + + send = &ic->i_sends[pos]; + first = send; + prev = NULL; + scat = &rm->m_sg[sg]; + sent = 0; + i = 0; + + /* Sometimes you want to put a fence between an RDMA + * READ and the following SEND. + * We could either do this all the time + * or when requested by the user. Right now, we let + * the application choose. + */ + if (rm->m_rdma_op && rm->m_rdma_op->r_fence) + send_flags = IB_SEND_FENCE; + + /* + * We could be copying the header into the unused tail of the page. + * That would need to be changed in the future when those pages might + * be mapped userspace pages or page cache pages. So instead we always + * use a second sge and our long-lived ring of mapped headers. We send + * the header after the data so that the data payload can be aligned on + * the receiver. + */ + + /* handle a 0-len message */ + if (be32_to_cpu(rm->m_inc.i_hdr.h_len) == 0) { + rds_iwarp_xmit_populate_wr(ic, send, pos, 0, 0, send_flags); + goto add_header; + } + + /* if there's data reference it with a chain of work reqs */ + for(; i < work_alloc && scat != &rm->m_sg[rm->m_count]; i++) { + unsigned int len; + + send = &ic->i_sends[pos]; + + len = min(RDS_FRAG_SIZE, ib_sg_dma_len(dev, scat) - off); + rds_iwarp_xmit_populate_wr(ic, send, pos, + ib_sg_dma_address(dev, scat) + off, len, + send_flags); + + /* + * We want to delay signaling completions just enough to get + * the batching benefits but not so much that we create dead time on the wire. + */ + if (ic->i_unsignaled_wrs-- == 0) { + ic->i_unsignaled_wrs = rds_iwarp_sysctl_max_unsig_wrs; + send->s_wr.send_flags |= IB_SEND_SIGNALED | IB_SEND_SOLICITED; + } + + ic->i_unsignaled_bytes -= len; + if (ic->i_unsignaled_bytes <= 0) { + ic->i_unsignaled_bytes = rds_iwarp_sysctl_max_unsig_bytes; + send->s_wr.send_flags |= IB_SEND_SIGNALED | IB_SEND_SOLICITED; + } + + rdsdebug("send %p wr %p num_sge %u next %p\n", send, + &send->s_wr, send->s_wr.num_sge, send->s_wr.next); + + sent += len; + off += len; + if (off == ib_sg_dma_len(dev, scat)) { + scat++; + off = 0; + } + +add_header: + /* Tack on the header after the data. The header SGE should already + * have been set up to point to the right header buffer. */ + memcpy(&ic->i_send_hdrs[pos], &rm->m_inc.i_hdr, sizeof(struct rds_header)); + + if (0) { + struct rds_header *hdr = &ic->i_send_hdrs[pos]; + + printk(KERN_NOTICE "send WR dport=%u flags=0x%x len=%d\n", + be16_to_cpu(hdr->h_dport), + hdr->h_flags, + be32_to_cpu(hdr->h_len)); + } + if (adv_credits) { + struct rds_header *hdr = &ic->i_send_hdrs[pos]; + + /* add credit and redo the header checksum */ + hdr->h_credit = adv_credits; + rds_message_make_checksum(hdr); + adv_credits = 0; + rds_iwarp_stats_inc(s_iwarp_tx_credit_updates); + } + + if (prev) + prev->s_wr.next = &send->s_wr; + prev = send; + + pos = (pos + 1) % ic->i_send_ring.w_nr; + } + + /* Account the RDS header in the number of bytes we sent, but just once. + * The caller has no concept of fragmentation. */ + if (hdr_off == 0) + sent += sizeof(struct rds_header); + + /* if we finished the message then send completion owns it */ + if (scat == &rm->m_sg[rm->m_count]) { + prev->s_rm = ic->i_rm; + prev->s_wr.send_flags |= IB_SEND_SIGNALED | IB_SEND_SOLICITED; + ic->i_rm = NULL; + } + + if (i < work_alloc) { + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc - i); + work_alloc = i; + } + if (ic->i_flowctl && i < credit_alloc) + rds_iwarp_send_add_credits(conn, credit_alloc - i); + + /* XXX need to worry about failed_wr and partial sends. */ + failed_wr = &first->s_wr; + ret = ib_post_send(ic->i_cm_id->qp, &first->s_wr, &failed_wr); + rdsdebug("ic %p first %p (wr %p) ret %d wr %p\n", ic, + first, &first->s_wr, ret, failed_wr); + BUG_ON(failed_wr != &first->s_wr); + if (ret) { + printk(KERN_WARNING "RDS/IWARP: ib_post_send to %u.%u.%u.%u " + "returned %d\n", NIPQUAD(conn->c_faddr), ret); + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc); + if (prev->s_rm) { + ic->i_rm = prev->s_rm; + prev->s_rm = NULL; + } + /* Finesse this later */ + BUG(); + goto out; + } + + ret = sent; +out: + BUG_ON(adv_credits); + return ret; +} + +int rds_iwarp_xmit_rdma(struct rds_connection *conn, struct rds_rdma_op *op) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + struct rds_iwarp_send_work *send = NULL; + struct rds_iwarp_send_work *first; + struct rds_iwarp_send_work *prev; + struct ib_send_wr *failed_wr; + struct rds_iwarp_device *rds_iwarpdev; + struct scatterlist *scat; + unsigned long len; + u64 remote_addr = op->r_remote_addr; + u32 pos; + u32 work_alloc; + u32 i; + u32 j; + int sent; + int ret; + int num_sge; + + rds_iwarpdev = ib_get_client_data(ic->i_cm_id->device, &rds_iwarp_client); + + /* map the message the first time we see it */ + op->r_count = ib_dma_map_sg(ic->i_cm_id->device, + op->r_sg, op->r_nents, (op->r_write) ? + DMA_TO_DEVICE : DMA_FROM_DEVICE); + rdsdebug("ic %p mapping op %p: %d\n", ic, op, op->r_count); + if (op->r_count == 0) { + rds_iwarp_stats_inc(s_iwarp_tx_sg_mapping_failure); + ret = -ENOMEM; /* XXX ? */ + goto out; + } + + /* + * Instead of knowing how to return a partial rdma read/write we insist that there + * be enough work requests to send the entire message. + */ + i = ceil(op->r_count, rds_iwarpdev->max_sge); + + work_alloc = rds_iwarp_ring_alloc(&ic->i_send_ring, i, &pos); + if (work_alloc != i) { + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc); + rds_iwarp_stats_inc(s_iwarp_tx_ring_full); + ret = -ENOMEM; + goto out; + } + + send = &ic->i_sends[pos]; + first = send; + prev = NULL; + scat = &op->r_sg[0]; + sent = 0; + num_sge = op->r_count; + + for ( i = 0; i < work_alloc && scat != &op->r_sg[op->r_count]; i++ ) { + send->s_wr.send_flags = 0; + send->s_queued = jiffies; + /* + * We want to delay signaling completions just enough to get + * the batching benefits but not so much that we create dead time on the wire. + */ + if (ic->i_unsignaled_wrs-- == 0) { + ic->i_unsignaled_wrs = rds_iwarp_sysctl_max_unsig_wrs; + send->s_wr.send_flags = IB_SEND_SIGNALED; + } + + send->s_wr.opcode = op->r_write ? IB_WR_RDMA_WRITE : IB_WR_RDMA_READ; + send->s_wr.wr.rdma.remote_addr = remote_addr; + send->s_wr.wr.rdma.rkey = op->r_key; + send->s_op = NULL; + + if (num_sge > rds_iwarpdev->max_sge) { + send->s_wr.num_sge = rds_iwarpdev->max_sge; + num_sge -= rds_iwarpdev->max_sge; + } + else + send->s_wr.num_sge = num_sge; + + send->s_wr.next = NULL; + + if (prev) + prev->s_wr.next = &send->s_wr; + + for (j = 0; j < send->s_wr.num_sge && scat != &op->r_sg[op->r_count]; j++) { + len = sg_dma_len(scat); + send->s_sge[j].addr = sg_dma_address(scat); + send->s_sge[j].length = len; + send->s_sge[j].lkey = ic->i_mr->lkey; + + sent += len; + rdsdebug("ic %p sent %d remote_addr %llu\n", ic, sent, remote_addr); + + remote_addr += sg_dma_len(scat); + scat++; + } + + rdsdebug("send %p wr %p num_sge %u next %p\n", send, + &send->s_wr, send->s_wr.num_sge, send->s_wr.next); + + prev = send; + if (++send == &ic->i_sends[ic->i_send_ring.w_nr]) + send = ic->i_sends; + } + + /* if we finished the message then send completion owns it */ + if (scat == &op->r_sg[op->r_count]) { + prev->s_wr.send_flags = IB_SEND_SIGNALED; + prev->s_op = op; + } + + if (i < work_alloc) { + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc - i); + work_alloc = i; + } + + failed_wr = &first->s_wr; + ret = ib_post_send(ic->i_cm_id->qp, &first->s_wr, &failed_wr); + rdsdebug("ic %p first %p (wr %p) ret %d wr %p\n", ic, + first, &first->s_wr, ret, failed_wr); + BUG_ON(failed_wr != &first->s_wr); + if (ret) { + printk(KERN_WARNING "RDS/IWARP: rdma ib_post_send to %u.%u.%u.%u " + "returned %d\n", NIPQUAD(conn->c_faddr), ret); + rds_iwarp_ring_unalloc(&ic->i_send_ring, work_alloc); + goto out; + } + + if (unlikely(failed_wr != &first->s_wr)) { + printk(KERN_WARNING "RDS/IWARP: ib_post_send() rc=%d, but failed_wqe updated!\n", ret); + BUG_ON(failed_wr != &first->s_wr); + } + + +out: + return ret; +} + +void rds_iwarp_xmit_complete(struct rds_connection *conn) +{ + struct rds_iwarp_connection *ic = conn->c_transport_data; + + /* We may have a pending ACK or window update we were unable + * to send previously (due to flow control). Try again. */ + rds_iwarp_attempt_ack(ic); +} diff --git a/net/rds/iwarp_stats.c b/net/rds/iwarp_stats.c new file mode 100644 index 0000000..192b328 --- /dev/null +++ b/net/rds/iwarp_stats.c @@ -0,0 +1,95 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include +#include +#include + +#include "rds.h" +#include "iwarp.h" + +RDS_DEFINE_PER_CPU(struct rds_iwarp_statistics, rds_iwarp_stats) + ____cacheline_aligned; + +static char *rds_iwarp_stat_names[] = { + "iwarp_connect_raced", + "iwarp_listen_closed_stale", + "iwarp_tx_cq_call", + "iwarp_tx_cq_event", + "iwarp_tx_ring_full", + "iwarp_tx_throttle", + "iwarp_tx_sg_mapping_failure", + "iwarp_tx_stalled", + "iwarp_tx_credit_updates", + "iwarp_rx_cq_call", + "iwarp_rx_cq_event", + "iwarp_rx_ring_empty", + "iwarp_rx_refill_from_cq", + "iwarp_rx_refill_from_thread", + "iwarp_rx_alloc_limit", + "iwarp_rx_credit_updates", + "iwarp_ack_sent", + "iwarp_ack_send_failure", + "iwarp_ack_send_delayed", + "iwarp_ack_send_piggybacked", + "iwarp_ack_received", + "iwarp_rdma_mr_alloc", + "iwarp_rdma_mr_free", + "iwarp_rdma_mr_used", + "iwarp_rdma_mr_pool_flush", + "iwarp_rdma_mr_pool_wait", +}; + +unsigned int rds_iwarp_stats_info_copy(struct rds_info_iterator *iter, + unsigned int avail) +{ + struct rds_iwarp_statistics stats = {0, }; + unsigned long *src; + unsigned long *sum; + size_t i; + int cpu; + + if (avail < ARRAY_SIZE(rds_iwarp_stat_names)) + goto out; + + for_each_online_cpu(cpu) { + src = (unsigned long *)&(rds_per_cpu(rds_iwarp_stats, cpu)); + sum = (unsigned long *)&stats; + for (i = 0; i < sizeof(stats) / sizeof(unsigned long); i++) + *(sum++) += *(src++); + } + + rds_stats_info_copy(iter, (unsigned long *)&stats, rds_iwarp_stat_names, + ARRAY_SIZE(rds_iwarp_stat_names)); +out: + return ARRAY_SIZE(rds_iwarp_stat_names); +} diff --git a/net/rds/iwarp_sysctl.c b/net/rds/iwarp_sysctl.c new file mode 100644 index 0000000..dd9060c --- /dev/null +++ b/net/rds/iwarp_sysctl.c @@ -0,0 +1,166 @@ +/* + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ +#include +#include +#include + +#include "iwarp.h" + +static struct ctl_table_header *rds_iwarp_sysctl_hdr; + +unsigned long rds_iwarp_sysctl_max_send_wr = RDS_IWARP_DEFAULT_SEND_WR; +unsigned long rds_iwarp_sysctl_max_recv_wr = RDS_IWARP_DEFAULT_RECV_WR; +unsigned long rds_iwarp_sysctl_max_recv_allocation = (128 * 1024 * 1024) / RDS_FRAG_SIZE; +static unsigned long rds_iwarp_sysctl_max_wr_min = 1; +/* hardware will fail CQ creation long before this */ +static unsigned long rds_iwarp_sysctl_max_wr_max = (u32)~0; + +unsigned long rds_iwarp_sysctl_max_unsig_wrs = 16; +static unsigned long rds_iwarp_sysctl_max_unsig_wr_min = 1; +static unsigned long rds_iwarp_sysctl_max_unsig_wr_max = 64; + +unsigned long rds_iwarp_sysctl_max_unsig_bytes = (16 << 20); +static unsigned long rds_iwarp_sysctl_max_unsig_bytes_min = 1; +static unsigned long rds_iwarp_sysctl_max_unsig_bytes_max = ~0UL; + +unsigned int rds_iwarp_sysctl_flow_control = 1; + +ctl_table rds_iwarp_sysctl_table[] = { + { + .ctl_name = 1, + .procname = "max_send_wr", + .data = &rds_iwarp_sysctl_max_send_wr, + .maxlen = sizeof(unsigned long), + .mode = 0644, + .proc_handler = &proc_doulongvec_minmax, + .extra1 = &rds_iwarp_sysctl_max_wr_min, + .extra2 = &rds_iwarp_sysctl_max_wr_max, + }, + { + .ctl_name = 2, + .procname = "max_recv_wr", + .data = &rds_iwarp_sysctl_max_recv_wr, + .maxlen = sizeof(unsigned long), + .mode = 0644, + .proc_handler = &proc_doulongvec_minmax, + .extra1 = &rds_iwarp_sysctl_max_wr_min, + .extra2 = &rds_iwarp_sysctl_max_wr_max, + }, + { + .ctl_name = 3, + .procname = "max_unsignaled_wr", + .data = &rds_iwarp_sysctl_max_unsig_wrs, + .maxlen = sizeof(unsigned long), + .mode = 0644, + .proc_handler = &proc_doulongvec_minmax, + .extra1 = &rds_iwarp_sysctl_max_unsig_wr_min, + .extra2 = &rds_iwarp_sysctl_max_unsig_wr_max, + }, + { + .ctl_name = 4, + .procname = "max_unsignaled_bytes", + .data = &rds_iwarp_sysctl_max_unsig_bytes, + .maxlen = sizeof(unsigned long), + .mode = 0644, + .proc_handler = &proc_doulongvec_minmax, + .extra1 = &rds_iwarp_sysctl_max_unsig_bytes_min, + .extra2 = &rds_iwarp_sysctl_max_unsig_bytes_max, + }, + { + .ctl_name = 5, + .procname = "max_recv_allocation", + .data = &rds_iwarp_sysctl_max_recv_allocation, + .maxlen = sizeof(unsigned long), + .mode = 0644, + .proc_handler = &proc_doulongvec_minmax, + }, + { + .ctl_name = 6, + .procname = "flow_control", + .data = &rds_iwarp_sysctl_flow_control, + .maxlen = sizeof(rds_iwarp_sysctl_flow_control), + .mode = 0644, + .proc_handler = &proc_dointvec, + }, + { .ctl_name = 0} +}; + +static ctl_table rds_iwarp_sysctl_rds_table[] = { + { + .ctl_name = SYSCTL_NET_RDS_IWARP, + .procname = "iwarp", + .data = NULL, + .maxlen = 0, + .mode = 0555, + .child = rds_iwarp_sysctl_table, + }, + { .ctl_name = 0} +}; + +static ctl_table rds_iwarp_sysctl_net_table[] = { + { + .ctl_name = SYSCTL_NET_RDS, + .procname = "rds", + .data = NULL, + .maxlen = 0, + .mode = 0555, + .child = rds_iwarp_sysctl_rds_table + }, + { .ctl_name = 0} +}; + +static ctl_table rds_iwarp_sysctl_root_table[] = { + { + .ctl_name = CTL_NET, + .procname = "net", + .data = NULL, + .maxlen = 0, + .mode = 0555, + .child = rds_iwarp_sysctl_net_table + }, + { .ctl_name = 0 } +}; + +void rds_iwarp_sysctl_exit(void) +{ + if (rds_iwarp_sysctl_hdr) + unregister_sysctl_table(rds_iwarp_sysctl_hdr); +} + +int __init rds_iwarp_sysctl_init(void) +{ + rds_iwarp_sysctl_hdr = register_sysctl_table(rds_iwarp_sysctl_root_table, 1); + if (rds_iwarp_sysctl_hdr == NULL) + return -ENOMEM; + return 0; +} diff --git a/net/rds/rdma.c b/net/rds/rdma.c index 1f1039e..9dbcdf5 100644 --- a/net/rds/rdma.c +++ b/net/rds/rdma.c @@ -258,7 +258,7 @@ static int __rds_rdma_map(struct rds_sock *rs, struct rds_get_mr_args *args, * Note that dma_map() implies that pending writes are * flushed to RAM, so no dma_sync is needed here. */ trans_private = rs->rs_transport->get_mr(sg, nents, - rs->rs_bound_addr, + rs, &mr->r_key); if (IS_ERR(trans_private)) { diff --git a/net/rds/rds.h b/net/rds/rds.h index 03031e2..b49a643 100644 --- a/net/rds/rds.h +++ b/net/rds/rds.h @@ -26,8 +26,9 @@ * * port 18633 was the version that had ack frames on the wire. */ -#define RDS_PORT 18634 - +#define RDS_TCP_PORT 18634 +#define RDS_IB_PORT 18635 +#define RDS_IWARP_PORT 18636 #ifndef AF_RDS #define AF_RDS 28 /* Reliable Datagram Socket */ @@ -58,6 +59,7 @@ /* XXX crap, we need to worry about this conflicting too */ #define SYSCTL_NET_RDS 9912 #define SYSCTL_NET_RDS_IB 100 +#define SYSCTL_NET_RDS_IWARP 101 #ifdef DEBUG #define rdsdebug(fmt, args...) pr_debug("%s(): " fmt, __func__ , ##args) @@ -375,7 +377,7 @@ struct rds_transport { unsigned int avail); void (*exit)(void); void *(*get_mr)(struct scatterlist *sg, unsigned long nr_sg, - __be32 ip_addr, u32 *key_ret); + struct rds_sock *rs, u32 *key_ret); void (*sync_mr)(void *trans_private, int direction); void (*free_mr)(void *trans_private, int invalidate); void (*flush_mrs)(void); diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c index 0389a99..298e372 100644 --- a/net/rds/tcp_connect.c +++ b/net/rds/tcp_connect.c @@ -96,7 +96,7 @@ int rds_tcp_conn_connect(struct rds_connection *conn) dest.sin_family = AF_INET; dest.sin_addr.s_addr = (__force u32)conn->c_faddr; - dest.sin_port = (__force u16)htons(RDS_PORT); + dest.sin_port = (__force u16)htons(RDS_TCP_PORT); /* * once we call connect() we can start getting callbacks and they diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c index caeacbe..50709b7 100644 --- a/net/rds/tcp_listen.c +++ b/net/rds/tcp_listen.c @@ -159,7 +159,7 @@ int __init rds_tcp_listen_init(void) sin.sin_family = PF_INET, sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); - sin.sin_port = (__force u16)htons(RDS_PORT); + sin.sin_port = (__force u16)htons(RDS_TCP_PORT); ret = sock->ops->bind(sock, (struct sockaddr *)&sin, sizeof(sin)); if (ret < 0) diff --git a/net/rds/transport.c b/net/rds/transport.c index 19a1a57..dd571c5 100644 --- a/net/rds/transport.c +++ b/net/rds/transport.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006 Oracle. All rights reserved. + * Copyright (c) 2008 Chelsio, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU @@ -163,6 +164,14 @@ int __init rds_trans_init(void) goto out; } #endif +#ifdef CONFIG_RDS_IWARP + { + extern int __init rds_iwarp_init(void); + ret = rds_iwarp_init(); + if (ret) + goto out; + } +#endif #ifdef CONFIG_RDS_TCP { extern int __init rds_tcp_init(void); From or.gerlitz at gmail.com Thu Jul 3 14:40:42 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Fri, 4 Jul 2008 00:40:42 +0300 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: References: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> <486CF681.6050206@voltaire.com> Message-ID: <15ddcffd0807031440t58c4fb8bqd39f3961d85c46e0@mail.gmail.com> On 7/4/08, Roland Dreier wrote: > Hmm, I missed the original email. > Anyway I have no idea how to do the block multicast feature really. I thought there was > some idea that userspace would pass QP creation flags into the kernel? yes, this was one idea (earlier one that is a new QP type eg IB_QPT_UD_BML) and the thing is that we would be happy to get your take on how can this be done with minimum pain regarding ABI issues both up towards libibverbs consumers and down towards ib_uverbs. Actually, looking on the XRC patch set, as it adds new QP type anyway, IB_QPT_UD_BML might turn to be the simplest way here, but its not general enough as moving forward, people were talking on other create flags they want to use from user space (low latency QP, etc) > Was there some reason that XRC wanted to use QP creation flags too? Among other things, the XRC patch series adds the "ibv_create_xrc_rcv_qp" new verb to libibverbs, so my thought here was that a new way X to create QPs is added, and we want a new way Y for the block multicast loop. So maybe we can form a framework Z for creating QPs other then the old one. Alternatively its possible for the create_qp_with_create_flags verb (and other verbs we want to add such as cq_modify_moderation_params) to just live next to the new verbs added to libibverbs in the new ibv_xrc_ops structure which is added for the verbs context. thoughts? Or. From rdreier at cisco.com Thu Jul 3 14:54:30 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 14:54:30 -0700 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <15ddcffd0807031440t58c4fb8bqd39f3961d85c46e0@mail.gmail.com> (Or Gerlitz's message of "Fri, 4 Jul 2008 00:40:42 +0300") References: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> <486CF681.6050206@voltaire.com> <15ddcffd0807031440t58c4fb8bqd39f3961d85c46e0@mail.gmail.com> Message-ID: > yes, this was one idea (earlier one that is a new QP type eg > IB_QPT_UD_BML) and the thing is that we would be happy to get your > take on how can this be done with minimum pain regarding ABI issues > both up towards libibverbs consumers and down towards ib_uverbs. Sorry, I don't have a magic solution for you. To minimize pain about ABI I think the only thing to do is minimize ABI changes. > Actually, looking on the XRC patch set, as it adds new QP type anyway, > IB_QPT_UD_BML might turn to be the simplest way here, but its not > general enough as moving forward, people were talking on other create > flags they want to use from user space (low latency QP, etc) If a QP creation flag works for XRC then I would prefer not to add more QP types I guess. One of the ways that the QP creation flags were originally sold to me was that XRC needed them anyway. So I'm a bit confused about where things have ended up. > Among other things, the XRC patch series adds the > "ibv_create_xrc_rcv_qp" new verb to libibverbs, so my thought here > was that a new way X to create QPs is added, and we want a new way Y > for the block multicast loop. So maybe we can form a framework Z for > creating QPs other then the old one. Alternatively its possible for > the create_qp_with_create_flags verb (and other verbs we want to add > such as cq_modify_moderation_params) to just live next to the new > verbs added to libibverbs in the new ibv_xrc_ops structure which is > added for the verbs context. I don't want to add both an XRC-specific mechanism and a more general mechanism that XRC could have used but doesn't. So yes the "framework Z" approach that works for both seems like the best way forward. - R. From swise at opengridcomputing.com Thu Jul 3 14:59:02 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 03 Jul 2008 16:59:02 -0500 Subject: [ofa-general] [PATCH for-2.6.27] RDMA/cxgb3 READ WR local_inv field uninitialized for normal reads. Message-ID: <20080703215902.20616.88793.stgit@dell3.ogc.int> From: Steve Wise Roland, you can merge this into the cxgb3 MEM_MGT_EXTENSIONS commit if you want. Steve. --- RDMA/cxgb3 READ WR local_inv field uninitialized for normal reads. Bug introduced by my read-with-inv changes. Need to set the local_inv field either way. Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/iwch_qp.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c index 0526a6c..87ac970 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_qp.c +++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c @@ -131,6 +131,8 @@ static int build_rdma_read(union t3_wr *wqe, struct ib_send_wr *wr, wqe->read.rdmaop = T3_READ_REQ; if (wr->opcode == IB_WR_RDMA_READ_WITH_INV) wqe->read.local_inv = 1; + else + wqe->read.local_inv = 0; wqe->read.reserved[0] = 0; wqe->read.reserved[1] = 0; wqe->read.rem_stag = cpu_to_be32(wr->wr.rdma.rkey); From or.gerlitz at gmail.com Thu Jul 3 14:59:08 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Fri, 4 Jul 2008 00:59:08 +0300 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> On 7/4/08, Jon Mason wrote: > The bulk of the code is a copy of the RDS IB with the changes necessary > to get iWARP working. So the intension is to have two code bases "RDS/IB" and "RDS/iWARP"? if yes, why? the rds code was enhanced to support credit management and my understanding is that this paves the way to have joint code, as done for any other RDMA ULP that was ported to iWARP (rNFS, Open-MPI, Lustre, etc) > The logic in the RDS connection setup to determine which method to use, > requires a new function in the core IB logic to translate a given IP > Address to the IB/iWARP device associated with it. The only place this > can be determined is in the pre-existing structs in cma.c. .laddr_check is not used in this patchset, and I could not understand its role from this description. > Olaf, if you would like to pull this into your personal tree, I can > provide you with delta patches until it is accepted into the OFED 1.4 > tree (assuming that you have no major problem with the code below). As its an RFC, lets have it go a little review before pulling it, unless the intension is to put it in an experimental branch and let people play with. Anyway, structured set of delta patches is always prefered, for review, acceptance, debugging and maintainace reasons. > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > +struct ib_device *ipaddr_to_ibdev(u32 addr) is there a reason why rdma_resolve_addr(NULL, addr, timeout) wouldn't do what ipaddr_to_ibdev does? Also, from quick looking I understand that this patchset adds some sort of fmr pool for the new fmrs within the rds code, can this pool be generalized and added to the core module such that other ULPs would be able to enjoy it? Or. From or.gerlitz at gmail.com Thu Jul 3 15:04:50 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Fri, 4 Jul 2008 01:04:50 +0300 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: References: <3b5e77ad0806300419w5c291b01ibb9b5a2ecd08efea@mail.gmail.com> <486CF681.6050206@voltaire.com> <15ddcffd0807031440t58c4fb8bqd39f3961d85c46e0@mail.gmail.com> Message-ID: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> On 7/4/08, Roland Dreier wrote: > Sorry, I don't have a magic solution for you. To minimize pain about > ABI I think the only thing to do is minimize ABI changes. fair-enough, so lets just try to take ABI changes to minimum... > If a QP creation flag works for XRC then I would prefer not to add more > QP types I guess. One of the ways that the QP creation flags were > originally sold to me was that XRC needed them anyway. So I'm a bit > confused about where things have ended up. I can't take this, we think that creation flags can be a good way to go wrt minimizing ABI changes, I am not enough into the XRC impl to tell why creation flags are not used there. > I don't want to add both an XRC-specific mechanism and a more general > mechanism that XRC could have used but doesn't. So yes the "framework > Z" approach that works for both seems like the best way forward. OK, so do we agree that ABI wise one thing to handle is the new verb/s plus their associated data structures (in our case just create_flags field added to qp_init_attr) and second thing is placing the new verbs into the verbs context structure? Lets see what Jack says, we are fine with anything (yes) he would suggest. Or. From rdreier at cisco.com Thu Jul 3 15:06:06 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 03 Jul 2008 15:06:06 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27] RDMA/cxgb3 READ WR local_inv field uninitialized for normal reads. In-Reply-To: <20080703215902.20616.88793.stgit@dell3.ogc.int> (Steve Wise's message of "Thu, 03 Jul 2008 16:59:02 -0500") References: <20080703215902.20616.88793.stgit@dell3.ogc.int> Message-ID: > Roland, you can merge this into the cxgb3 MEM_MGT_EXTENSIONS commit > if you want. OK, done. master.kernel.org is down for maintenance, so I can't push and you won't see it for a while. From olaf.kirch at oracle.com Thu Jul 3 15:09:45 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 00:09:45 +0200 Subject: [ofa-general] Re: [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <200807040009.47794.olaf.kirch@oracle.com> On Thursday 03 July 2008 23:34:12 Jon Mason wrote: > This patch adds support for running RDS over iWARP adapters. It > requires the fast_reg_mr patches and Olaf's RDS patches (for the flow > control enablement) to apply cleanly and build. It also includes the > patch I sent out earlier to have different connection ports for IB, TCP, > and IWARP. > > It is a "work in progress", but I would like for people to take a look Thanks for posting the code at this stage. I will have a closer look tomorrow and see how much of the code it duplicates from ib*.c actually could be shared. As Or already noted, it would be preferable to have IB and iWARP share as much as possible. I understand there probably needs to be a different transport class for iWARP, but I would like iWARP to be some kind of "we do everything like IB except these two little bits" transport. > The bulk of the code is a copy of the RDS IB with the changes necessary > to get iWARP working. Thus there may be stale comments and redundant > code. I will audit the code to remove the stale comments, and remove > duplicated and unmodified code (but feel free to point out any that you > see). I'll go over it tomorrow and see how much can be accomodated in the existing transport interface, and what needs additional function pointers. > Olaf, if you would like to pull this into your personal tree, I can > provide you with delta patches until it is accepted into the OFED 1.4 > tree (assuming that you have no major problem with the code below). Thanks, let's see how far we get with breaking this down into pieces. Okay? Regards, Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From ralph.campbell at qlogic.com Thu Jul 3 16:02:07 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 03 Jul 2008 16:02:07 -0700 Subject: [ofa-general] [PATCH] IB/ipath - Change the vendor_id reported by ibv_query_device() Message-ID: <20080703230207.466.91454.stgit@eng-46.mv.qlogic.com> The IB spec. for SubnGet(NodeInfo) and query HCA says that the vendor ID field should be the IEEE OUI assigned to the vendor. The PCI vendor ID was being returned instead. This will affect applications which call ibv_query_device(). The old value was 0x001fc1 or 0x001077, the new value is 0x001175. The vendor ID doesn't appear to be exported via /sys so that should reduce possible compatibility issues. I'm only aware of OpenMPI as a major application which depends on this change. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_mad.c | 6 +++--- drivers/infiniband/hw/ipath/ipath_verbs.c | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_mad.c b/drivers/infiniband/hw/ipath/ipath_mad.c index 5f9315d..be4fc9a 100644 --- a/drivers/infiniband/hw/ipath/ipath_mad.c +++ b/drivers/infiniband/hw/ipath/ipath_mad.c @@ -111,9 +111,9 @@ static int recv_subn_get_nodeinfo(struct ib_smp *smp, nip->revision = cpu_to_be32((majrev << 16) | minrev); nip->local_port_num = port; vendor = dd->ipath_vendorid; - nip->vendor_id[0] = 0; - nip->vendor_id[1] = vendor >> 8; - nip->vendor_id[2] = vendor; + nip->vendor_id[0] = IPATH_SRC_OUI_1; + nip->vendor_id[1] = IPATH_SRC_OUI_2; + nip->vendor_id[2] = IPATH_SRC_OUI_3; return reply(smp); } diff --git a/drivers/infiniband/hw/ipath/ipath_verbs.c b/drivers/infiniband/hw/ipath/ipath_verbs.c index 7779165..9e23ab0 100644 --- a/drivers/infiniband/hw/ipath/ipath_verbs.c +++ b/drivers/infiniband/hw/ipath/ipath_verbs.c @@ -1497,7 +1497,8 @@ static int ipath_query_device(struct ib_device *ibdev, IB_DEVICE_SYS_IMAGE_GUID | IB_DEVICE_RC_RNR_NAK_GEN | IB_DEVICE_PORT_ACTIVE_EVENT | IB_DEVICE_SRQ_RESIZE; props->page_size_cap = PAGE_SIZE; - props->vendor_id = dev->dd->ipath_vendorid; + props->vendor_id = + IPATH_SRC_OUI_1 << 16 | IPATH_SRC_OUI_2 << 8 | IPATH_SRC_OUI_3; props->vendor_part_id = dev->dd->ipath_deviceid; props->hw_ver = dev->dd->ipath_pcirev; From sean.hefty at intel.com Thu Jul 3 17:15:05 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 3 Jul 2008 17:15:05 -0700 Subject: [ofa-general] RE: [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <000001c8dd6b$02a7f520$8398070a@amr.corp.intel.com> >+struct ib_device *ipaddr_to_ibdev(u32 addr) >+{ >+ struct rdma_dev_addr dev_addr; >+ struct cma_device *cma_dev; >+ struct net_device *dev; >+ union ib_gid gid; >+ >+ dev = ip_dev_find(addr); >+ if (!dev) >+ goto out; >+ >+ rdma_copy_addr(&dev_addr, dev, NULL); >+ >+ switch (dev->type) { >+ case ARPHRD_INFINIBAND: >+ ib_addr_get_sgid(&dev_addr, &gid); >+ break; >+ case ARPHRD_ETHER: >+ iw_addr_get_sgid(&dev_addr, &gid); >+ break; >+ default: >+ goto out; >+ } >+ >+ list_for_each_entry(cma_dev, &dev_list, list) { >+ int ret; >+ u8 port; >+ >+ ret = ib_find_cached_gid(cma_dev->device, &gid, &port, NULL); >+ if (!ret) >+ return cma_dev->device; >+ } >+out: >+ return NULL; >+} >+EXPORT_SYMBOL(ipaddr_to_ibdev); You should be able to use either rdma_resolve_addr or rdma_bind_addr (if this is a local address) to get what you need. Alternately, look at the ib_addr module for lower-level calls. - Sean From edpaternal at csiionline.com Thu Jul 3 20:49:45 2008 From: edpaternal at csiionline.com (Bobbye Kellan) Date: Fri, 4 Jul 2008 05:49:45 +0200 Subject: [ofa-general] Be strong and assured Message-ID: <001301c8dd99$c32bc6e0$015cfa2c@LeanDJ1> Open new world of healthy living Buy any medicine that you need here -------------- next part -------------- An HTML attachment was scrubbed... URL: From kadir.ozaltin2 at mail.dcu.ie Thu Jul 3 22:10:35 2008 From: kadir.ozaltin2 at mail.dcu.ie (Irish Lottery News Center) Date: Thu, 3 Jul 2008 22:10:35 -0700 Subject: [ofa-general] Final Notification/Confirm Your Winning Amount Message-ID: <4858FDD100016951@hawk.dcu.ie> Ref: UK/9420X2/68 Batch: 074/05/ZY369 This is to inform you that you have been selected for a cash prize of £750,000.00(GBP)incash credited to file KTU/9023118308/03 held on the month of June 2008. To claim your prize,Contact Mr Derek White with the following informations. Name,Address,Age,Occupation,Country Email:irn.customerunit at live.com Mr Clark Johnson From jwnkindred at bcds.com.au Thu Jul 3 23:04:51 2008 From: jwnkindred at bcds.com.au (Terrie Mccollum) Date: Fri, 4 Jul 2008 01:04:51 -0500 Subject: [ofa-general] Do not refuse you in plesure Message-ID: <001b01c8dd71$f631c940$0491e804@CABINA10> You donts seem to need blue pilules or do you? Opening door to a world full of sensual delights! Here! exclamdown etc/auditd epineurial exhibiters epichordal fautorship erzeugungs entry/exit ettiquette eprofessor fLinfinity fBrmntstat -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Jul 3 23:49:54 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 4 Jul 2008 09:49:54 +0300 Subject: [ofa-general] Issues with osm_sm_state_mgr_check_legality() In-Reply-To: <486C90BD.5040807@bull.net> References: <486C90BD.5040807@bull.net> Message-ID: <20080704064954.GF2319@sashak.voltaire.com> Hi Vincent, On 10:41 Thu 03 Jul , Vincent Ficet wrote: > Hello, > > In opensm/opensm/osm_sm_state_mgr.c:osm_sm_state_mgr_check_legality(), I > noticed that the only valid signals that the SM can get while in the > IB_SMINFO_STATE MASTER are: > - OSM_SM_SIGNAL_HANDOVER > - OSM_SM_SIGNAL_HANDOVER_SENT > > However, in opensm/opensm/osm_sm_state_mgr.c:osm_sm_state_mgr_process(), we > see that the following signals can be received in the master state: > - OSM_SM_SIGNAL_POLLING_TIMEOUT > - OSM_SM_SIGNAL_HANDOVER > - OSM_SM_SIGNAL_HANDOVER_SENT > - OSM_SM_SIGNAL_WAIT_FOR_HANDOVER > - OSM_SM_SIGNAL_DISCOVER > > Is this a bug, or am I missing anything here ? I don't see an immediate bug here. osm_sm_state_mgr_check_legality() is used only during SMInfo request processing so it covers only this flow path and osm_sm_state_mgr_process() is called in different places. Sasha From sashak at voltaire.com Thu Jul 3 23:51:24 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 4 Jul 2008 09:51:24 +0300 Subject: [ofa-general] Re: [OPENSM][TRIVIAL] fix true/false usage In-Reply-To: <1214934841.16508.19.camel@cardanus.llnl.gov> References: <1214934841.16508.19.camel@cardanus.llnl.gov> Message-ID: <20080704065124.GG2319@sashak.voltaire.com> On 10:54 Tue 01 Jul , Al Chu wrote: > Hey Sasha, > > Just noticed that true/false was not used consistently for some > variables. > > Al > > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > From eb84a35076476167d7612bedd95f0648e29b0e69 Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Tue, 1 Jul 2008 10:35:52 -0700 > Subject: [PATCH] fix true/false usage > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From sashak at voltaire.com Thu Jul 3 23:51:40 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 4 Jul 2008 09:51:40 +0300 Subject: [ofa-general] Re: [OPENSM][TRIVIAL] Fix comment typo In-Reply-To: <1214938011.16508.21.camel@cardanus.llnl.gov> References: <1214938011.16508.21.camel@cardanus.llnl.gov> Message-ID: <20080704065140.GH2319@sashak.voltaire.com> On 11:46 Tue 01 Jul , Al Chu wrote: > Hey Sasha, > > Typo I noticed. Nothing fancy. > > Al > > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > From 826858a586a85dfa761223ed216e66fca9d26ce7 Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Tue, 1 Jul 2008 11:44:28 -0700 > Subject: [PATCH] fix typo - force_immediate_heavy_sweep is now force_heavy_sweep > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From sashak at voltaire.com Fri Jul 4 00:11:12 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 4 Jul 2008 10:11:12 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop In-Reply-To: <20080701183305.586a6146.weiny2@llnl.gov> References: <20080701183305.586a6146.weiny2@llnl.gov> Message-ID: <20080704071112.GI2319@sashak.voltaire.com> Hi Ira, Some nit... On 18:33 Tue 01 Jul , Ira Weiny wrote: > From 7a9131f07ffa29e509004fb9835a82eca93d3fac Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 1 Jul 2008 14:31:00 -0700 > Subject: [PATCH] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop > > > Signed-off-by: Ira K. Weiny > --- > infiniband-diags/src/saquery.c | 97 +++++++++++++++------------------------- > 1 files changed, 36 insertions(+), 61 deletions(-) > > diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c > index 1594cad..922703f 100644 > --- a/infiniband-diags/src/saquery.c > +++ b/infiniband-diags/src/saquery.c > @@ -237,32 +237,11 @@ static void dump_path_record(void *data) > ); > } > > -/** > - * str must be longer than 32 to hold the full gid. > - * len will be checked to ensure this. > - */ > -static char * > -sprint_gid(ib_gid_t *gid, char *str, size_t len) > -{ > - int i = 0; > - char tmp[16]; > - > - assert(str != NULL); > - assert(len > 32); > - > - str[0] = '\0'; > - for (i = 0; i < 16; i++) { > - sprintf(tmp, "%02X", gid->raw[i]); > - strcat(str, tmp); > - } > - > - return (str); > -} > - > static void dump_class_port_info(void *data) > { > size_t GID_STR_LEN = 256; > char gid_str[GID_STR_LEN]; > + char gid_str2[GID_STR_LEN]; We can use INET6_ADDRSTRLEN as array size. > ib_class_port_info_t *class_port_info = data; > > printf("SA ClassPortInfo:\n" > @@ -271,13 +250,13 @@ static void dump_class_port_info(void *data) > "\t\tCapability mask..........0x%04X\n" > "\t\tCapability mask 2........0x%08X\n" > "\t\tResponse time value......0x%02X\n" > - "\t\tRedirect GID.............0x%s\n" > + "\t\tRedirect GID.............%s\n" > "\t\tRedirect TC/SL/FL........0x%08X\n" > "\t\tRedirect LID.............0x%04X\n" > "\t\tRedirect PKey............0x%04X\n" > "\t\tRedirect QP..............0x%08X\n" > "\t\tRedirect QKey............0x%08X\n" > - "\t\tTrap GID.................0x%s\n" > + "\t\tTrap GID.................%s\n" > "\t\tTrap TC/SL/FL............0x%08X\n" > "\t\tTrap LID.................0x%04X\n" > "\t\tTrap PKey................0x%04X\n" > @@ -289,13 +268,13 @@ static void dump_class_port_info(void *data) > cl_ntoh16(class_port_info->cap_mask), > ib_class_cap_mask2(class_port_info), > ib_class_resp_time_val(class_port_info), > - sprint_gid(&(class_port_info->redir_gid), gid_str, GID_STR_LEN), > + inet_ntop(AF_INET6, &(class_port_info->redir_gid), gid_str, GID_STR_LEN), and sizeof(gid_str) instead of GID_STR_LEN variable. Sasha > cl_ntoh32(class_port_info->redir_tc_sl_fl), > cl_ntoh16(class_port_info->redir_lid), > cl_ntoh16(class_port_info->redir_pkey), > cl_ntoh32(class_port_info->redir_qp), > cl_ntoh32(class_port_info->redir_qkey), > - sprint_gid(&(class_port_info->trap_gid), gid_str, GID_STR_LEN), > + inet_ntop(AF_INET6, &(class_port_info->trap_gid), gid_str2, GID_STR_LEN), > cl_ntoh32(class_port_info->trap_tc_sl_fl), > cl_ntoh16(class_port_info->trap_lid), > cl_ntoh16(class_port_info->trap_pkey), > @@ -326,20 +305,20 @@ static void dump_portinfo_record(void *data) > > static void dump_multicast_group_record(void *data) > { > + size_t GID_STR_LEN = 256; > + char gid_str[GID_STR_LEN]; > ib_member_rec_t *p_mcmr = data; > uint8_t sl; > ib_member_get_sl_flow_hop(p_mcmr->sl_flow_hop, &sl, NULL, NULL); > printf("MCMemberRecord group dump:\n" > - "\t\tMGID....................0x%016" PRIx64 " : " > - "0x%016" PRIx64 "\n" > + "\t\tMGID....................%s\n" > "\t\tMlid....................0x%X\n" > "\t\tMtu.....................0x%X\n" > "\t\tpkey....................0x%X\n" > "\t\tRate....................0x%X\n" > "\t\tSL......................0x%X\n" > "", > - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), > - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, GID_STR_LEN), > cl_ntoh16( p_mcmr->mlid ), > p_mcmr->mtu, > cl_ntoh16( p_mcmr->pkey ), > @@ -350,8 +329,10 @@ static void dump_multicast_group_record(void *data) > > static void dump_multicast_member_record(void *data) > { > + size_t GID_STR_LEN = 256; > + char gid_str[GID_STR_LEN]; > + char gid_str2[GID_STR_LEN]; > ib_member_rec_t *p_mcmr = data; > - uint64_t gid_prefix = cl_ntoh64( p_mcmr->port_gid.unicast.prefix ); > uint64_t gid_interface_id = cl_ntoh64( p_mcmr->port_gid.unicast.interface_id ); > uint16_t mlid = cl_ntoh16( p_mcmr->mlid ); > int i = 0; > @@ -371,29 +352,26 @@ static void dump_multicast_member_record(void *data) > > if (requested_name) { > if (strtol(requested_name, NULL, 0) == mlid) { > - printf("\t\tPortGid.................0x%016" PRIx64 " : " > - "0x%016" PRIx64 " (%s)\n", > - gid_prefix, > - gid_interface_id, > + printf("\t\tPortGid.................%s (%s)\n", > + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, > + gid_str, GID_STR_LEN), > node_name > ); > } > } else { > printf("MCMemberRecord member dump:\n" > - "\t\tMGID....................0x%016" PRIx64 " : " > - "0x%016" PRIx64 "\n" > + "\t\tMGID....................%s\n" > "\t\tMlid....................0x%X\n" > - "\t\tPortGid.................0x%016" PRIx64 " : " > - "0x%016" PRIx64 "\n" > + "\t\tPortGid.................%s\n" > "\t\tScopeState..............0x%X\n" > "\t\tProxyJoin...............0x%X\n" > "\t\tNodeDescription.........%s\n" > "", > - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), > - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, > + GID_STR_LEN), > cl_ntoh16( p_mcmr->mlid ), > - gid_prefix, > - gid_interface_id, > + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, > + gid_str2, GID_STR_LEN), > p_mcmr->scope_state, > p_mcmr->proxy_join, > node_name > @@ -403,6 +381,8 @@ static void dump_multicast_member_record(void *data) > > static void dump_service_record(void *data) > { > + size_t GID_STR_LEN = 256; > + char gid_str[GID_STR_LEN]; > char buf_service_key[35]; > char buf_service_name[65]; > ib_service_record_t *p_sr = data; > @@ -430,8 +410,7 @@ static void dump_service_record(void *data) > > printf("ServiceRecord dump:\n" > "\t\tServiceID...............0x%016" PRIx64 "\n" > - "\t\tServiceGID..............0x%016" PRIx64 " : " > - "0x%016" PRIx64 "\n" > + "\t\tServiceGID..............%s\n" > "\t\tServiceP_Key............0x%X\n" > "\t\tServiceLease............0x%X\n" > "\t\tServiceKey..............%s\n" > @@ -468,8 +447,7 @@ static void dump_service_record(void *data) > "\t\tServiceData64.2.........0x%016" PRIx64 "\n" > "", > cl_ntoh64( p_sr->service_id ), > - cl_ntoh64( p_sr->service_gid.unicast.prefix ), > - cl_ntoh64( p_sr->service_gid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_sr->service_gid.raw, gid_str, GID_STR_LEN), > cl_ntoh16( p_sr->service_pkey ), > cl_ntoh32( p_sr->service_lease ), > buf_service_key, > @@ -501,6 +479,9 @@ static void dump_service_record(void *data) > > static void dump_inform_info_record(void *data) > { > + size_t GID_STR_LEN = 256; > + char gid_str[GID_STR_LEN]; > + char gid_str2[GID_STR_LEN]; > ib_inform_info_record_t *p_iir = data; > uint32_t qpn; > uint8_t resp_time_val; > @@ -510,11 +491,10 @@ static void dump_inform_info_record(void *data) > if (p_iir->inform_info.is_generic) { > printf("InformInfoRecord dump:\n" > "\t\tRID\n" > - "\t\tSubscriberGID...........0x%016" PRIx64 " : " > - "0x%016" PRIx64 "\n" > + "\t\tSubscriberGID...........%s\n" > "\t\tSubscriberEnum..........0x%X\n" > "\t\tInformInfo dump:\n" > - "\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" > + "\t\tgid.....................%s\n" > "\t\tlid_range_begin.........0x%X\n" > "\t\tlid_range_end...........0x%X\n" > "\t\tis_generic..............0x%X\n" > @@ -525,11 +505,9 @@ static void dump_inform_info_record(void *data) > "\t\tresp_time_val...........0x%X\n" > "\t\tnode_type...............0x%06X\n" > "", > - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), > - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, gid_str, GID_STR_LEN), > cl_ntoh16( p_iir->subscriber_enum ), > - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), > - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, gid_str2, GID_STR_LEN), > cl_ntoh16( p_iir->inform_info.lid_range_begin ), > cl_ntoh16( p_iir->inform_info.lid_range_end ), > p_iir->inform_info.is_generic, > @@ -543,11 +521,10 @@ static void dump_inform_info_record(void *data) > } else { > printf("InformInfoRecord dump:\n" > "\t\tRID\n" > - "\t\tSubscriberGID...........0x%016" PRIx64 " : " > - "0x%016" PRIx64 "\n" > + "\t\tSubscriberGID...........%s\n" > "\t\tSubscriberEnum..........0x%X\n" > "\t\tInformInfo dump:\n" > - "\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" > + "\t\tgid.....................%s\n" > "\t\tlid_range_begin.........0x%X\n" > "\t\tlid_range_end...........0x%X\n" > "\t\tis_generic..............0x%X\n" > @@ -558,11 +535,9 @@ static void dump_inform_info_record(void *data) > "\t\tresp_time_val...........0x%X\n" > "\t\tvendor_id...............0x%06X\n" > "", > - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), > - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, gid_str, GID_STR_LEN), > cl_ntoh16( p_iir->subscriber_enum ), > - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), > - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), > + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, gid_str2, GID_STR_LEN), > cl_ntoh16( p_iir->inform_info.lid_range_begin ), > cl_ntoh16( p_iir->inform_info.lid_range_end ), > p_iir->inform_info.is_generic, > -- > 1.5.4.5 > From olaf.kirch at oracle.com Fri Jul 4 00:29:01 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 09:29:01 +0200 Subject: [ofa-general] Re: [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <200807040929.02476.olaf.kirch@oracle.com> Hi Jon, If we make it so that iWARP and IB share the same rds_transport, is there still a need to use a different port for iWARP than for IB? Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From jean-vincent.ficet at bull.net Fri Jul 4 00:34:31 2008 From: jean-vincent.ficet at bull.net (Vincent Ficet) Date: Fri, 04 Jul 2008 09:34:31 +0200 Subject: [ofa-general] Issues with osm_sm_state_mgr_check_legality() In-Reply-To: <20080704064954.GF2319@sashak.voltaire.com> References: <486C90BD.5040807@bull.net> <20080704064954.GF2319@sashak.voltaire.com> Message-ID: <486DD287.1050003@bull.net> Hi Sasha, Sasha Khapyorsky wrote: > Hi Vincent, > > On 10:41 Thu 03 Jul , Vincent Ficet wrote: > >> Hello, >> >> In opensm/opensm/osm_sm_state_mgr.c:osm_sm_state_mgr_check_legality(), I >> noticed that the only valid signals that the SM can get while in the >> IB_SMINFO_STATE MASTER are: >> - OSM_SM_SIGNAL_HANDOVER >> - OSM_SM_SIGNAL_HANDOVER_SENT >> >> However, in opensm/opensm/osm_sm_state_mgr.c:osm_sm_state_mgr_process(), we >> see that the following signals can be received in the master state: >> - OSM_SM_SIGNAL_POLLING_TIMEOUT >> - OSM_SM_SIGNAL_HANDOVER >> - OSM_SM_SIGNAL_HANDOVER_SENT >> - OSM_SM_SIGNAL_WAIT_FOR_HANDOVER >> - OSM_SM_SIGNAL_DISCOVER >> >> Is this a bug, or am I missing anything here ? >> > > I don't see an immediate bug here. osm_sm_state_mgr_check_legality() is > used only during SMInfo request processing so it covers only this flow > path and osm_sm_state_mgr_process() is called in different places. > > OK, I see, Thanks for the tip. I have another question: I compiled opensm and ran it from a different prefix. Because I'm not root, I wanted opensm to log onto a path that was not /var/cache. At first, I forgot to set the OSM_CACHE_DIR environment variable and got the following error: Error from osm_opensm_init: IB_INSUFFICIENT_RESOURCES. Looking at strace, I quickly found what was wrong: lstat("/var/cache/opensm/", 0x7ffff7cbf970) = -1 ENOENT (No such file or directory) mkdir("/var/cache/opensm/", 0755) = -1 EACCES (Permission denied) Shouldn't opensm log something more accurate w.r.t this issue ? Strictly speaking, it's not really an IB_INSUFFICIENT_RESOURCES issue. Cheers, Vincent From olaf.kirch at Oracle.com Fri Jul 4 01:54:05 2008 From: olaf.kirch at Oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 10:54:05 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <200807041054.06497.olaf.kirch@oracle.com> > +int rds_iwarp_flush_mr_pool(struct rds_iwarp_mr_pool *pool, int free_all) > +{ ... > + spin_lock_irqsave(&pool->list_lock, flags); > + /* Get the list of all MRs to be dropped. Ordering matters - > + * we want to put drop_list ahead of free_list. */ > + list_splice_init(&pool->free_list, &unmap_list); > + list_splice_init(&pool->drop_list, &unmap_list); > + if (free_all) > + list_splice_init(&pool->clean_list, &unmap_list); > + spin_unlock_irqrestore(&pool->list_lock, flags); > + > + free_goal = rds_iwarp_flush_goal(pool, free_all); > + > + if (list_empty(&unmap_list)) > + goto out; > + > + ib_dereg_mr(ibmr->fastreg_mr); This probably cannot work. ibmr is uninitialized at this point. The ib_rdma code uses ib_unmap_fmr(&fmr_list) which does a batched unmap. If iWARP doesn't have an equivalent mechanism, we might as well destroy each iwarp MR immediately when it is released. OTOH if we want to go for maximum code sharing, one could put the ib_dereg_mr call into __rds_iwarp_teardown_mr, so that it gets cleaned up in the following loop: > + /* Now we can destroy the DMA mapping and unpin any pages */ > + list_for_each_entry_safe(ibmr, next, &unmap_list, list) { > + unpinned += ibmr->sg_len; > + __rds_iwarp_teardown_mr(ibmr); > + if (nfreed < free_goal || ibmr->remap_count >= pool->fmr_attr.max_maps) { > + rds_iwarp_stats_inc(s_iwarp_rdma_mr_free); > + list_del(&ibmr->list); > + kfree(ibmr); > + nfreed++; > + } > + ncleaned++; > + } Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From olaf.kirch at oracle.com Fri Jul 4 01:59:54 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 10:59:54 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <200807041059.54661.olaf.kirch@oracle.com> On Thursday 03 July 2008 23:34:12 Jon Mason wrote: > + ibmr->fastreg_mr = ib_alloc_fast_reg_mr(rds_iwarpdev->pd, ibmr->page_list->max_page_list_len); > + if (IS_ERR(ibmr->page_list)) { > + err = PTR_ERR(ibmr->fastreg_mr); > + ibmr->fastreg_mr = NULL; > + printk(KERN_WARNING "RDS/IWARP: ib_alloc_fast_reg_mr failed (err=%d)\n", err); > + goto out_no_cigar; > + } You actually want to check IS_ERR(ibmr->fastreg_mr) here. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From Fabian_Salamanca at Dell.com Fri Jul 4 02:07:48 2008 From: Fabian_Salamanca at Dell.com (Fabian_Salamanca at Dell.com) Date: Fri, 4 Jul 2008 04:07:48 -0500 Subject: [ofa-general] Mellanox HCA issue (Dell) In-Reply-To: <200807041059.54661.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041059.54661.olaf.kirch@oracle.com> Message-ID: <68EBC1CE8929234284877A97158910D1014D3EB5@MEXX3M01.mex.amer.dell.com> Hi! I have a cluster with mellanox HCA's, OFED 1.3 and an embedded Cisco SFS in a Dell blade enclosure, I can't enable my HCA's: [root at compute-0-12 bin]# ofed_info OFED-1.3.1 libibverbs: git://git.openfabrics.org/ofed_1_3/libibverbs.git ofed_1_3 commit 40b771aa6a9c0ad092b2e20775b4723d3b173792 libmthca: git://git.openfabrics.org/ofed_1_3/libmthca.git ofed_1_3 commit 9501e698d257949acfab2edc90812602966dbcc9 libmlx4: git://git.openfabrics.org/ofed_1_3/libmlx4.git ofed_1_3 commit 3869d6dab7e12fe452270ca641f7dd7082b42482 libehca: git://git.openfabrics.org/ofed_1_3/libehca.git ofed_1_3 commit fd898180cfa3b737f893f432a80b91bac3396325 libipathverbs: git://git.openfabrics.org/ofed_1_3/libipathverbs.git ofed_1_3 commit 82be4d81859d1fd2edf830220fe65a9923b80a46 libcxgb3: git://git.openfabrics.org/ofed_1_3/libcxgb3.git ofed_1_3 commit 6f7485feb244d8571fcab2292ef92c97bea48df0 libnes: git://git.openfabrics.org/ofed_1_3/libnes.git ofed_1_3 commit 471fa2e5a7bb2f8946119396358c31adcc6c2fb3 libibcm: git://git.openfabrics.org/ofed_1_3/libibcm.git ofed_1_3 commit 53ec35f544bbc1838bbadc2210909c25a954a5e2 librdmacm: git://git.openfabrics.org/ofed_1_3/librdmacm.git ofed_1_3 commit a0ef80a1e0d5debdae48a844fbc8d09aec5b24b1 dapl1: git://git.openfabrics.org/ofed_1_3/dapl1.git ofed_1_3 commit 7a9b58d6c50fc0a357de540ec3eb2ab2e07f8779 dapl2: git://git.openfabrics.org/ofed_1_3/dapl2.git ofed_1_3 commit 2583f07d9d0f55eee14e0b0e6074bc6fd0712177 libsdp: git://git.openfabrics.org/ofed_1_3/libsdp.git ofed_1_3 commit c8102dccc502930442b23de658674d386456b350 sdpnetstat: git://git.openfabrics.org/ofed_1_3/sdpnetstat.git ofed_1_3 commit 3341620a7259c4f7bdd4180864b98e260c3dc223 srptools: git://git.openfabrics.org/ofed_1_3/srptools.git ofed_1_3 commit e0ce2d42eeb25f8e89b8f6daaa32a630c9b64f0d perftest: git://git.openfabrics.org/ofed_1_3/perftest.git ofed_1_3 commit 6321b5468f7293088cc003809049c02b176130d8 qlvnictools: git://git.openfabrics.org/ofed_1_3/qlvnictools.git ofed_1_3 commit 086f9cb80ee790d61bddaf201ecbae32a2ff21dd tvflash: git://git.openfabrics.org/ofed_1_3/tvflash.git ofed_1_3 commit f5e7407a7f2058448df5e5320d9843f944427429 mstflint: git://git.openfabrics.org/ofed_1_3/mstflint.git ofed_1_3 commit 78bbd3d521a9078553a991111ffb6f76665b9ee9 qperf: git://git.openfabrics.org/ofed_1_3/qperf.git ofed_1_3 commit 6221aabd038df0b7033e035378ca190641ed2295 management: git://git.openfabrics.org/ofed_1_3/management.git ofed_1_3 commit d9c852406dae14e8284f9cfb1c7f495bbb55fddf ibutils: git://git.openfabrics.org/ofed_1_3/ibutils.git ofed_1_3 commit 7daf94fab6eaf307316326f3f49704e6080a1508 ibsim: git://git.openfabrics.org/ofed_1_3/ibsim.git ofed_1_3 commit 55113d9f919709c7c97ea41d29991941b9c8be70 ofa_kernel-1.3.1: Git: git://git.openfabrics.org/ofed_1_3/linux-2.6.git ofed_kernel commit 39e1dc833f98e5134f91fcf7f33df402adf4bc0c # MPI mvapich-1.0.1-2533.src.rpm mvapich2-1.0.3-1.src.rpm openmpi-1.2.6-1.src.rpm mpitests-3.0-773.src.rpm [root at compute-0-12 bin]# ibstat CA 'mlx4_0' CA type: MT25418 Number of ports: 2 Firmware version: 2.3.0 Hardware version: a0 Node GUID: 0x00188b9097fe3885 System image GUID: 0x00188b9097fe3888 Port 1: State: Initializing Physical state: LinkUp Rate: 20 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x02510868 Port GUID: 0x00188b9097fe3886 Port 2: State: Down Physical state: Polling Rate: 10 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x02510868 Port GUID: 0x00188b9097fe3887 [root at compute-0-12 bin]# ibv_devinfo -v hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0018:8b90:97fe:3885 sys_image_guid: 0018:8b90:97fe:3888 vendor_id: 0x02c9 vendor_part_id: 25418 hw_ver: 0xA0 board_id: DEL08C0000001 phys_port_cnt: 2 max_mr_size: 0xffffffffffffffff page_size_cap: 0xfffff000 max_qp: 131008 max_qp_wr: 16351 device_cap_flags: 0x001c1c66 max_sge: 32 max_sge_rd: 0 max_cq: 65408 max_cqe: 4194303 max_mr: 131056 max_pd: 32764 max_qp_rd_atom: 16 max_ee_rd_atom: 0 max_res_rd_atom: 2096128 max_qp_init_rd_atom: 128 max_ee_init_rd_atom: 0 atomic_cap: ATOMIC_HCA (1) max_ee: 0 max_rdd: 0 max_mw: 0 max_raw_ipv6_qp: 0 max_raw_ethy_qp: 0 max_mcast_grp: 8192 max_mcast_qp_attach: 56 max_total_mcast_qp_attach: 458752 max_ah: 0 max_fmr: 0 max_srq: 65472 max_srq_wr: 16383 max_srq_sge: 31 max_pkeys: 128 local_ca_ack_delay: 15 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 0 port_lmc: 0x00 max_msg_sz: 0x40000000 port_cap_flags: 0x02510868 max_vl_num: 8 (4) bad_pkey_cntr: 0x0 qkey_viol_cntr: 0x0 sm_sl: 0 pkey_tbl_len: 128 gid_tbl_len: 128 subnet_timeout: 0 init_type_reply: 0 active_width: 4X (2) active_speed: 5.0 Gbps (2) phys_state: LINK_UP (5) GID[ 0]: fe80:0000:0000:0000:0018:8b90:97fe:3886 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 0 port_lmc: 0x00 max_msg_sz: 0x40000000 port_cap_flags: 0x02510868 max_vl_num: 8 (4) bad_pkey_cntr: 0x0 qkey_viol_cntr: 0x0 sm_sl: 0 pkey_tbl_len: 128 gid_tbl_len: 128 subnet_timeout: 0 init_type_reply: 0 active_width: 4X (2) active_speed: 2.5 Gbps (1) phys_state: POLLING (2) GID[ 0]: fe80:0000:0000:0000:0018:8b90:97fe:3887 What can I do? Thanks!! From olaf.kirch at oracle.com Fri Jul 4 02:13:16 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 11:13:16 +0200 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807041054.06497.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041054.06497.olaf.kirch@oracle.com> Message-ID: <200807041113.16872.olaf.kirch@oracle.com> On Friday 04 July 2008 10:54:05 Olaf Kirch wrote: > OTOH if we want to go for maximum code sharing, one could put the > ib_dereg_mr call into __rds_iwarp_teardown_mr, so that it gets cleaned > up in the following loop: Okay, I re-read the fastreg patch. So it seems what you want to do is to allocate the fastreg_mr once, and remap it several times. Correct? Then this fits the current model quite well, except that we need to splice the whole thing into the send cq handling (so that we get told when the MR was mapped). Am I supposed to use the send WQ for mapping the MR, or could one use a separate WQ - so that we avoid stalling the MR mapping behind a large queue of send WRs? Anyway, I looked at your code and tried to fold it back into the IB code. It seems except for the RDMA code there's not a huge difference there. I'll post a patch based on your work later today Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From dotanba at gmail.com Fri Jul 4 03:35:01 2008 From: dotanba at gmail.com (Dotan Barak) Date: Fri, 04 Jul 2008 12:35:01 +0200 Subject: [ofa-general] Mellanox HCA issue (Dell) In-Reply-To: <68EBC1CE8929234284877A97158910D1014D3EB5@MEXX3M01.mex.amer.dell.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041059.54661.olaf.kirch@oracle.com> <68EBC1CE8929234284877A97158910D1014D3EB5@MEXX3M01.mex.amer.dell.com> Message-ID: <486DFCD5.6090202@gmail.com> Hi. Fabian_Salamanca at Dell.com wrote: > port: 1 > state: PORT_INIT (2) > phys_state: LINK_UP (5) > > port: 2 > state: PORT_DOWN (1) > phys_state: POLLING (2) > > > > What can I do? > It seems that port 1 of your HCA is connected to something but no subnet manager (SM) is running. Port 2 is not connected to anything (there isn't any cable connected...) Dotan From vlad at lists.openfabrics.org Fri Jul 4 02:40:22 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Fri, 4 Jul 2008 02:40:22 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080704-0200 daily build status Message-ID: <20080704094022.DFE83E60D89@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.26-rc6 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c: At top level: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: error: unknown field 'change_queue_depth' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080704-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From olaf.kirch at oracle.com Fri Jul 4 03:07:39 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 12:07:39 +0200 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807041113.16872.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041054.06497.olaf.kirch@oracle.com> <200807041113.16872.olaf.kirch@oracle.com> Message-ID: <200807041207.40490.olaf.kirch@oracle.com> On Friday 04 July 2008 11:13:16 Olaf Kirch wrote: > Okay, I re-read the fastreg patch. So it seems what you want to do is to > allocate the fastreg_mr once, and remap it several times. Correct? > Then this fits the current model quite well, except that we need > to splice the whole thing into the send cq handling (so that we > get told when the MR was mapped). There's another problem with the approach chosen here. The iwarp rdma code will try to post the two WRs directly to the send WQ. This will confuse the book-keeping done in ib_send.c, where we assume we can post new WRs to the send queue as long as we're able to allocate WRs from the send ring. (Note how we set up the send WQ having a size of the send ring + 1 - where the +1 is for the ACK WR). So if you just post WRs to the send CQ, you create two problems - 1. your ib_post_send may fail, so you need to deal with this 2. someone else's ib_post_send in ib_send.c may fail, and that will confuse the code greatly. So you really need to go through the ib_send code and allocate WRs there, and wait if you can't get any... that gets icky fast. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From olaf.kirch at oracle.com Fri Jul 4 04:46:51 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 4 Jul 2008 13:46:51 +0200 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807041207.40490.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041113.16872.olaf.kirch@oracle.com> <200807041207.40490.olaf.kirch@oracle.com> Message-ID: <200807041346.52282.olaf.kirch@oracle.com> Oops. There's an even bigger problem I didn't notice until now. You pass the rds socket into get_mr in order to get a rds_conn and the QP. Because you need a QP to post your fastreg WR to. However, that doesn't work the way you approach it. The rs_conn inside the rds_socket is just a cached pointer. It's what we got when we called rds_conn_lookup in the most recent call to rds_sendmsg - we do this for efficiency reasons, because it helps us avoid taking a spinlock. So in other words, rds_sock->rs_conn may be NULL, it may point to some random unrelated rds_connection - and in some cases it may even point to the rds_conn you're interested in. We have two choices here - change the GET_MR API so that the caller includes the destination IP - then we can always locate the correct conn. - require that iWARP RDMA only works with the new cmsg based interface; in this case the MR mapping happens from inside sendmsg, where we naturally have the destination address Neither of these will make the Oracle IPC layer happy. AFAIK Oracle IPC doesn't pass around the destination IP when allocating a MR using GET_MR. And it doesn't support the new get_mr interface either. To what degree are we bound to a specific QP? In other words, if I want to do RDMA over a given QP, do I have to perform the fastreg over this specific QP, or could I use any QP? So, would it work to create a QP specifically for the purpose of performing fastregs? If a fastreg is bound to a specific QP, then what happens if the RDS connection bounces and we have to allocate a fresh QP? Would that render all existing fastregs invalid? Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From pw at osc.edu Fri Jul 4 15:07:43 2008 From: pw at osc.edu (Pete Wyckoff) Date: Fri, 4 Jul 2008 18:07:43 -0400 Subject: [ofa-general] Re: [PATCH] iw cm conn sent destroy In-Reply-To: <486D3A21.2030205@opengridcomputing.com> References: <20080703192556.GA1179@osc.edu> <20080703203754.GC1304@osc.edu> <486D3A21.2030205@opengridcomputing.com> Message-ID: <20080704220743.GA10165@osc.edu> swise at opengridcomputing.com wrote on Thu, 03 Jul 2008 15:44 -0500: > Pete Wyckoff wrote: >> sean.hefty at intel.com wrote on Thu, 03 Jul 2008 13:03 -0700: >> >>>> That makes sense. I couldn't find anyway to get that bit >>>> IWCM_F_CONNECT_WAIT cleared (without massive layering violations). >>>> Is there anything like rdma_give_up_on_the_connect() that works >>>> on the client side? Am I expected just to wait for the transport >>>> layer to time out the connect? I could imagine a Ctrl-C situation >>>> where immediate client-side cancellation is desired. >>>> >>> From the perspective of the rdma_cm interface, rdma_destroy_id() is >>> intended to provide this functionality. >>> >> >> That is a sensible approach. The implementation in iwcm.c just >> passively waits for a connection to time out before allowing >> rdma_destroy_id() to complete. Maybe I should look at adding a case >> in cma_cancel_operation() for CMA_CONNECT that calls into a >> device-specific function that can actively quit the connection. >> >> Or should this be a special case in iwcm's destroy_cm_id()? Where >> a CONN_SENT state will bypass the check on IWCM_F_CONNECT_WAIT, and >> call into a device function that tries to cancel the connection >> attepmt first? > > If we define a method that every iwarp provider supports to cancel a > connect attempt, then we can do away with the IWCM_F_CONNECT_WAIT bit > altogether and always cancel the operation instead. For cxgb3, the > connect request to the HW really cannot be canceled. So the driver > would have to keep its endpoint struct around until the HW finally posts > the connect attempt results, but mark the endpoint as "no cm_id > attached". The if the connection succeeds, the driver will immediately > reset it I guess... This defines an optional iwcm method that cancels the connect operation if the device supports that. Otherwise it will wait on the CONNECT_WAIT bit as usual. Doing it like this does not require changes in existing devices. It would not be pretty to implement a mandatory connect_cancel() as you describe for cxgb3. (I dropped the first-attempt patch in this thread.) -- Pete iwcm: connect cancel Add an interface to cancel a connection attempt. Used on the client side of rdmacm to cancel a connection, for devices that support such an operation. This can avoid waiting for the device to timeout the operation itself. Signed-off-by: Pete Wyckoff --- drivers/infiniband/core/iwcm.c | 14 ++++++++++++++ include/rdma/iw_cm.h | 2 ++ 2 files changed, 16 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index 81c9195..56aa0d6 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -327,6 +327,20 @@ static void destroy_cm_id(struct iw_cm_id *cm_id) int ret; cm_id_priv = container_of(cm_id, struct iwcm_id_private, id); + + /* + * Give the device a chance to cancel the connect, if possible. + * Else we wait until it times out. + */ + if (cm_id_priv->state == IW_CM_STATE_CONN_SENT && + cm_id->device->iwcm->connect_cancel) { + ret = cm_id->device->iwcm->connect_cancel(cm_id); + if (ret == 0) { + clear_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags); + cm_id_priv->state = IW_CM_STATE_IDLE; + } + } + /* * Wait if we're currently in a connect or accept downcall. A * listening endpoint should never block here. diff --git a/include/rdma/iw_cm.h b/include/rdma/iw_cm.h index aeefa9b..046690c 100644 --- a/include/rdma/iw_cm.h +++ b/include/rdma/iw_cm.h @@ -119,6 +119,8 @@ struct iw_cm_verbs { int (*connect)(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param); + int (*connect_cancel)(struct iw_cm_id *cm_id); + int (*accept)(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param); -- 1.5.5.1 From 1bb at earvin.com Fri Jul 4 22:14:59 2008 From: 1bb at earvin.com (car ludovic) Date: Sat, 05 Jul 2008 05:14:59 +0000 Subject: [ofa-general] to general Message-ID: <000901c8de6d$02e70cf9$53dcef98@mrwiyq> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avakwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From a-asrerb at 800branded.com Sat Jul 5 01:55:31 2008 From: a-asrerb at 800branded.com (Neva Mayes) Date: Sat, 5 Jul 2008 05:55:31 -0300 Subject: [ofa-general] message from dating website Message-ID: <01c8de63$bb908b80$41dcfac8@a-asrerb> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Lisa at GuideLo.info only, because I am using my friend's email to write this. Would you mind me showing some nice pictures of me? -------------- next part -------------- A non-text attachment was scrubbed... Name: me Type: image/gif Size: 41022 bytes Desc: not available URL: From vlad at lists.openfabrics.org Sat Jul 5 02:40:22 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sat, 5 Jul 2008 02:40:22 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080705-0200 daily build status Message-ID: <20080705094022.4F2ABE6024B@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c: At top level: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: error: unknown field 'change_queue_depth' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080705-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From psjacqueline.holton at assurant.com Sat Jul 5 05:22:44 2008 From: psjacqueline.holton at assurant.com (Jetzt SOFT bestellen) Date: Sat, 5 Jul 2008 21:22:44 +0900 Subject: [ofa-general] Eine sehr breite Auswahl aus den besten Softwaren Message-ID: <966705557.06444443996505@assurant.com> Laden Sie sofort unsere Software runter! Es handelt sich nur um Originalversionen, in allen europaeischen Sprachen. Wir haben guenstigste Preise und Softwareprodukte fuer PC und MAC! Unser kompetentes Team wird Ihnen bei der Istallation helfen, falls Sie es brauchen. Wir bieten Geld-Zurueck-Garantie und rasche Antworten vom Support! Bei uns bekommen Sie nur perfekte Software http://waewosoft.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From 5hanson at azoncanada.com Sat Jul 5 20:31:50 2008 From: 5hanson at azoncanada.com (kellen tiziano) Date: Sun, 06 Jul 2008 03:31:50 +0000 Subject: [ofa-general] to general Message-ID: <000701c8df27$05fc42ca$111aa8ae@txioa> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avakwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From vlad at dev.mellanox.co.il Sat Jul 5 23:35:49 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Sun, 06 Jul 2008 09:35:49 +0300 Subject: [ofa-general] ofa_kernel-1.3.1 backport for kernel 2.6.16-54-0.2.5_lustre.1.6.4.3-smp In-Reply-To: <39C75744D164D948A170E9792AF8E7CAC5AFD1@exil.voltaire.com> References: <39C75744D164D948A170E9792AF8E7CAC5AFD1@exil.voltaire.com> Message-ID: <487067C5.2050309@dev.mellanox.co.il> Moshe Kazir wrote: > Moshe > > > ____________________________________________________________ > > Moshe Katzir | +972-9971-8639 (o) | +972-52-860-6042 (m) > > > > Voltaire – _The Grid Backbone_ > > _ _ > > www.voltaire.com > > > > Applied for OFED-1.3.2 and OFED-1.4. Regards, Vladimir From jackm at dev.mellanox.co.il Sun Jul 6 00:16:28 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 6 Jul 2008 10:16:28 +0300 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> Message-ID: <200807061016.29228.jackm@dev.mellanox.co.il> On Friday 04 July 2008 01:04, Or Gerlitz wrote: > I can't take this, we think that creation flags can be a good way to > go wrt minimizing ABI changes, I am not enough into the XRC impl to > tell why creation flags are not used there. > XRC was added as a new QP type, because it is a separate, distinct qp type which allows an RC QP to send directly to an SRQ, rather than to an RC QP. Knowledge that we are dealing with XRC QPs needs to be maintained in the core layer beyond just QP creation time (e.g., ib_destroy_qp() needs this information to decrement the xrc domain usecount when an XRC QP is destroyed), so that using a creation flag for this is not clean. Regarding the creation flag for XRC RCV QPs, I think I can (and should) get rid of it. It is only used by ib_uverbs_create_xrc_rcv_qp() to indicate to the low-level driver that we're creating an xrc receive qp, not a regular xrc qp (the caller NEVER sets this flag). ib_uverbs_create_xrc_rcv_qp() also invokes the low-level driver's create_xrc_rcv_qp function, so that this flag is clearly unneeded (the low-level driver can "remember" that it is dealing with an XRC receive qp by just setting a flag internal to its own qp object, and not a qp creation flag. - Jack From eli at mellanox.co.il Sun Jul 6 01:25:35 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 6 Jul 2008 11:25:35 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: remove unused code Message-ID: <20080706082535.GA11165@mtls03> The IPOIB_MCAST_STARTED flag is not used at all so remove it. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 1 - drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 8 -------- 2 files changed, 0 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index df10fb0..634fba6 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -89,7 +89,6 @@ enum { IPOIB_FLAG_SUBINTERFACE = 5, IPOIB_MCAST_RUN = 6, IPOIB_STOP_REAPER = 7, - IPOIB_MCAST_STARTED = 8, IPOIB_FLAG_ADMIN_CM = 9, IPOIB_FLAG_UMCAST = 10, IPOIB_FLAG_CSUM = 11, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 4a6538b..0b7d129 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -592,10 +592,6 @@ int ipoib_mcast_start_thread(struct net_device *dev) queue_delayed_work(ipoib_workqueue, &priv->mcast_task, 0); mutex_unlock(&mcast_mutex); - spin_lock_irq(&priv->lock); - set_bit(IPOIB_MCAST_STARTED, &priv->flags); - spin_unlock_irq(&priv->lock); - return 0; } @@ -605,10 +601,6 @@ int ipoib_mcast_stop_thread(struct net_device *dev, int flush) ipoib_dbg_mcast(priv, "stopping multicast thread\n"); - spin_lock_irq(&priv->lock); - clear_bit(IPOIB_MCAST_STARTED, &priv->flags); - spin_unlock_irq(&priv->lock); - mutex_lock(&mcast_mutex); clear_bit(IPOIB_MCAST_RUN, &priv->flags); cancel_delayed_work(&priv->mcast_task); -- 1.5.6.1 From eli at mellanox.co.il Sun Jul 6 01:26:17 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 6 Jul 2008 11:26:17 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: remove priv->mcast_mutex Message-ID: <20080706082617.GA11207@mtls03> No need for a mutex when calling ib_attach_mcast/ib_detach_mcast since these operations are synchronized at the HW driver layer. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 1 - drivers/infiniband/ulp/ipoib/ipoib_main.c | 1 - drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 4 ---- 3 files changed, 0 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 634fba6..b5727f7 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -276,7 +276,6 @@ struct ipoib_dev_priv { unsigned long flags; - struct mutex mcast_mutex; struct mutex vlan_mutex; struct rb_root path_tree; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index b3fd7e8..8be9ea0 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1072,7 +1072,6 @@ static void ipoib_setup(struct net_device *dev) spin_lock_init(&priv->lock); spin_lock_init(&priv->tx_lock); - mutex_init(&priv->mcast_mutex); mutex_init(&priv->vlan_mutex); INIT_LIST_HEAD(&priv->path_list); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 96f9aa7..f50ebe0 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -61,9 +61,7 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid) } /* attach QP to multicast group */ - mutex_lock(&priv->mcast_mutex); ret = ib_attach_mcast(priv->qp, mgid, mlid); - mutex_unlock(&priv->mcast_mutex); if (ret) ipoib_warn(priv, "failed to attach to multicast group, ret = %d\n", ret); @@ -77,9 +75,7 @@ int ipoib_mcast_detach(struct net_device *dev, u16 mlid, union ib_gid *mgid) struct ipoib_dev_priv *priv = netdev_priv(dev); int ret; - mutex_lock(&priv->mcast_mutex); ret = ib_detach_mcast(priv->qp, mgid, mlid); - mutex_unlock(&priv->mcast_mutex); if (ret) ipoib_warn(priv, "ib_detach_mcast failed (result = %d)\n", ret); -- 1.5.6.1 From eli at mellanox.co.il Sun Jul 6 01:28:43 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 6 Jul 2008 11:28:43 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: set QP qkey once after joining broadcast group Message-ID: <20080706082843.GA11244@mtls03> The current code will set the qkey for any join of a none sendonly multicast group. The operation involves a modify QP operation and is only required after join of the broadcast group. The patch introduces a new parameter to ipoib_mcast_attach to control that. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 2 +- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 4 ++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 28 ++++++++++++----------- 3 files changed, 19 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index b5727f7..dc10449 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -484,7 +484,7 @@ void ipoib_path_iter_read(struct ipoib_path_iter *iter, #endif int ipoib_mcast_attach(struct net_device *dev, u16 mlid, - union ib_gid *mgid); + union ib_gid *mgid, int set_qkey); int ipoib_mcast_detach(struct net_device *dev, u16 mlid, union ib_gid *mgid); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 0b7d129..55ebd95 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -186,6 +186,7 @@ static int ipoib_mcast_join_finish(struct ipoib_mcast *mcast, struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_ah *ah; int ret; + int set_qkey = 0; mcast->mcmember = *mcmember; @@ -200,6 +201,7 @@ static int ipoib_mcast_join_finish(struct ipoib_mcast *mcast, priv->qkey = be32_to_cpu(priv->broadcast->mcmember.qkey); spin_unlock_irq(&priv->lock); priv->tx_wr.wr.ud.remote_qkey = priv->qkey; + set_qkey = 1; } if (!test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) { @@ -212,7 +214,7 @@ static int ipoib_mcast_join_finish(struct ipoib_mcast *mcast, } ret = ipoib_mcast_attach(dev, be16_to_cpu(mcast->mcmember.mlid), - &mcast->mcmember.mgid); + &mcast->mcmember.mgid, set_qkey); if (ret < 0) { ipoib_warn(priv, "couldn't attach QP to multicast group " IPOIB_GID_FMT "\n", diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index f50ebe0..ba7c886 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -33,18 +33,13 @@ #include "ipoib.h" -int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid) +int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey) { struct ipoib_dev_priv *priv = netdev_priv(dev); - struct ib_qp_attr *qp_attr; + struct ib_qp_attr *qp_attr = NULL; int ret; u16 pkey_index; - ret = -ENOMEM; - qp_attr = kmalloc(sizeof *qp_attr, GFP_KERNEL); - if (!qp_attr) - goto out; - if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &pkey_index)) { clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); ret = -ENXIO; @@ -52,12 +47,19 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid) } set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); - /* set correct QKey for QP */ - qp_attr->qkey = priv->qkey; - ret = ib_modify_qp(priv->qp, qp_attr, IB_QP_QKEY); - if (ret) { - ipoib_warn(priv, "failed to modify QP, ret = %d\n", ret); - goto out; + if (set_qkey) { + ret = -ENOMEM; + qp_attr = kmalloc(sizeof *qp_attr, GFP_KERNEL); + if (!qp_attr) + goto out; + + /* set correct QKey for QP */ + qp_attr->qkey = priv->qkey; + ret = ib_modify_qp(priv->qp, qp_attr, IB_QP_QKEY); + if (ret) { + ipoib_warn(priv, "failed to modify QP, ret = %d\n", ret); + goto out; + } } /* attach QP to multicast group */ -- 1.5.6.1 From vlad at lists.openfabrics.org Sun Jul 6 02:40:22 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sun, 6 Jul 2008 02:40:22 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080706-0200 daily build status Message-ID: <20080706094022.2F25DE60248@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.26-rc6 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c: At top level: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: error: unknown field 'change_queue_depth' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.c:552: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser/iscsi_iser.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband/ulp/iser] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080706-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Sun Jul 6 02:54:49 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 6 Jul 2008 12:54:49 +0300 (IDT) Subject: [ofa-general] loading/reading the value of qos_ca_high_limit Message-ID: Hi Sasha, I see now that when qos_ca_high_limit is set to zero, its missing among the prints that say what is loaded from the cache file. Note the first case where I have set it to zero vs the second case where it was one and the line "Loading Cached Option:qos_ca_high_limit" was present in the output. Also, is there a way to read the value of high_limit (and active number of VLs) from the node? smpquery vlarb does not seem to provide it. thanks, Or. # grep qos_ca /var/cache/opensm/opensm.opts qos_ca_max_vls 4 qos_ca_high_limit 0 qos_ca_vlarb_high 0:1 qos_ca_vlarb_low 0:32,1:96 qos_ca_sl2vl 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 # opensm -p 15 -Q -D 0x1 ------------------------------------------------- OpenSM 3.1.11 Reading Cached Option File: /var/cache/opensm//opensm.opts Loading Cached Option:guid = 0x0008f104039708d9 Loading Cached Option:partition_config_file = /home/ogerlitz/pkey.txt Loading Cached Option:sm_priority = 15 Loading Cached Option:node_name_map_name = (null) Loading Cached Option:log_flags = 255 Loading Cached Option:force_log_flush = TRUE Loading Cached Option:qos = TRUE Loading Cached Option:qos_policy_file = /home/ogerlitz/qos.txt.ok Loading Cached Option:qos_ca_max_vls = 4 Loading Cached Option:qos_ca_vlarb_high = 0:1 Loading Cached Option:qos_ca_vlarb_low = 0:32,1:96 Command Line Arguments: Priority = 15 verbose option -D = 0x1 Log File: /var/log/opensm.log ------------------------------------------------- OpenSM 3.1.11 Entering MASTER state SUBNET UP # grep qos_ca /var/cache/opensm/opensm.opts qos_ca_max_vls 4 qos_ca_high_limit 1 qos_ca_vlarb_high 0:1 qos_ca_vlarb_low 0:32,1:96 qos_ca_sl2vl 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 # opensm -p 15 -Q -D 0x1 ------------------------------------------------- OpenSM 3.1.11 Reading Cached Option File: /var/cache/opensm//opensm.opts Loading Cached Option:guid = 0x0008f104039708d9 Loading Cached Option:partition_config_file = /home/ogerlitz/pkey.txt Loading Cached Option:sm_priority = 15 Loading Cached Option:node_name_map_name = (null) Loading Cached Option:log_flags = 255 Loading Cached Option:force_log_flush = TRUE Loading Cached Option:qos = TRUE Loading Cached Option:qos_policy_file = /home/ogerlitz/qos.txt.ok Loading Cached Option:qos_ca_max_vls = 4 Loading Cached Option:qos_ca_high_limit = 1 Loading Cached Option:qos_ca_vlarb_high = 0:1 Loading Cached Option:qos_ca_vlarb_low = 0:32,1:96 Command Line Arguments: Priority = 15 verbose option -D = 0x1 Log File: /var/log/opensm.log ------------------------------------------------- OpenSM 3.1.11 Entering MASTER state SUBNET UP # /usr/local/sbin/smpquery vlarb 11 1 # VLArbitration tables: Lid 11 port 1 LowCap 8 HighCap 8 # Low priority VL Arbitration Table: VL : |0x0 |0x1 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | WEIGHT: |0x20|0x60|0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | # High priority VL Arbitration Table: VL : |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | WEIGHT: |0x1 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | From jackm at dev.mellanox.co.il Sun Jul 6 03:00:31 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 6 Jul 2008 13:00:31 +0300 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> Message-ID: <200807061300.31811.jackm@dev.mellanox.co.il> The ABI for ibv_create_qp (see file libibverbs/include/infiniband/kern-abi.h) currently has 1 reserved byte. If we can use this for create flags, we don't need to break the ABI, but clearly note that: a. we will have space for only 8 flag bits -- not very many, they seem to be going like hotcakes. b. Kernel space create flags are 32 bits -- it may be weird to have userspace flags be only 8 bits. We've also tried to keep bit offsets between kernel space and userspace the same (e.g., see device capabilities flags) -- and this would clearly make it impossible. On the other hand, I don't like the idea of adding a new verb for each new tweak. In the case of xrc receive qp's, new verbs were needed because the parameter list for XRC receive qp's was very different (XRC domain handle and QP num, instead of qp handle). In the case multicast loopback, a creation flag is most appropriate, because this is really a qp-creation-only issue (the kernel core layer only needs this info at QP creation time). Probably, the least short-term pain (no ABI change) would be to use the last reserved create-qp ABI byte for flags, while defining the flags field in ibv_qp_init_attr as uint32_t (preparing for more than 8 future flags). We would not have the same flag bits in user as in kernel, and if we at some point needed more that 8 user-space qp create flags, we would increment the ABI at that point, keep the originally defined 8 flags as they were, and just add a _u32 with additional flags. Adding the flags field to the qp_init_attr structure does, however, require incrementing the libibverbs version to 1.2, necessitating an IBVERBS_1.1 compatibility layer. - Jack On Friday 04 July 2008 01:04, Or Gerlitz wrote: > On 7/4/08, Roland Dreier wrote: > > > Sorry, I don't have a magic solution for you. To minimize pain about > > ABI I think the only thing to do is minimize ABI changes. > > fair-enough, so lets just try to take ABI changes to minimum... > > > If a QP creation flag works for XRC then I would prefer not to add more > > QP types I guess. One of the ways that the QP creation flags were > > originally sold to me was that XRC needed them anyway. So I'm a bit > > confused about where things have ended up. > > I can't take this, we think that creation flags can be a good way to > go wrt minimizing ABI changes, I am not enough into the XRC impl to > tell why creation flags are not used there. > > > > I don't want to add both an XRC-specific mechanism and a more general > > mechanism that XRC could have used but doesn't. So yes the "framework > > Z" approach that works for both seems like the best way forward. > > OK, so do we agree that ABI wise one thing to handle is the new verb/s > plus their associated data structures (in our case just create_flags > field added to qp_init_attr) and second thing is placing the new verbs > into the verbs context structure? > > Lets see what Jack says, we are fine with anything (yes) he would suggest. > > Or. > From hrosenstock at xsigo.com Sun Jul 6 04:13:22 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Sun, 06 Jul 2008 04:13:22 -0700 Subject: [ofa-general] loading/reading the value of qos_ca_high_limit In-Reply-To: References: Message-ID: <1215342803.3486.347.camel@hrosenstock-ws.xsigo.com> On Sun, 2008-07-06 at 12:54 +0300, Or Gerlitz wrote: > Also, is there a way to read the value of high_limit (and active > number > of VLs) from the node? smpquery vlarb does not seem to provide it. Those components are part of PortInfo attribute so smpquery portinfo has those. -- Hal From sashak at voltaire.com Sun Jul 6 04:23:33 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 6 Jul 2008 14:23:33 +0300 Subject: [ofa-general] Re: loading/reading the value of qos_ca_high_limit In-Reply-To: References: Message-ID: <20080706112333.GA12450@sashak.voltaire.com> Hi Or, Second question is first... On 12:54 Sun 06 Jul , Or Gerlitz wrote: > > Also, is there a way to read the value of high_limit (and active number > of VLs) from the node? smpquery vlarb does not seem to provide it. It is encoded in PortInfo (VLHighLimit and OperVLs): smpquery portinfo 3 # Port info: Lid 3 port 0 Mkey:............................0x0000000000000000 GidPrefix:.......................0xfe80000000000000 Lid:.............................0x0003 SMLid:...........................0x0001 CapMask:.........................0x500a68 IsTrapSupported IsAutomaticMigrationSupported IsSLMappingSupported IsLedInfoSupported IsSystemImageGUIDsupported IsVendorClassSupported IsCapabilityMaskNoticeSupported DiagCode:........................0x0000 MkeyLeasePeriod:.................30 LocalPort:.......................1 LinkWidthEnabled:................1X or 4X LinkWidthSupported:..............1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps LinkState:.......................Active PhysLinkState:...................LinkUp LinkDownDefState:................Polling ProtectBits:.....................1 LMC:.............................0 LinkSpeedActive:.................2.5 Gbps LinkSpeedEnabled:................2.5 Gbps NeighborMTU:.....................2048 SMSL:............................0 VLCap:...........................VL0-7 InitType:........................0x00 VLHighLimit:.....................0 VLArbHighCap:....................8 VLArbLowCap:.....................8 InitReply:.......................0x00 MtuCap:..........................2048 VLStallCount:....................7 HoqLife:.........................16 OperVLs:.........................VL0-7 PartEnforceInb:..................0 PartEnforceOutb:.................0 FilterRawInb:....................0 FilterRawOutb:...................0 MkeyViolations:..................0 PkeyViolations:..................0 QkeyViolations:..................0 GuidCap:.........................32 ClientReregister:................0 SubnetTimeout:...................18 RespTimeVal:.....................1 LocalPhysErr:....................8 OverrunErr:......................8 MaxCreditHint:...................0 RoundTrip:.......................0 Sasha From sashak at voltaire.com Sun Jul 6 04:27:51 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 6 Jul 2008 14:27:51 +0300 Subject: [ofa-general] Re: loading/reading the value of qos_ca_high_limit In-Reply-To: References: Message-ID: <20080706112751.GB12450@sashak.voltaire.com> Hi again Or, On 12:54 Sun 06 Jul , Or Gerlitz wrote: > > I see now that when qos_ca_high_limit is set to zero, its missing among > the prints that say what is loaded from the cache file. Note the first > case where I have set it to zero vs the second case where it was one and > the line "Loading Cached Option:qos_ca_high_limit" was present in the output. It is correct. "0" is a default value, so when you set it = 0 in a config file - it shows nothing (eg a value was not not changed from its default). Sasha From sashak at voltaire.com Sun Jul 6 04:43:42 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 6 Jul 2008 14:43:42 +0300 Subject: [ofa-general] Issues with osm_sm_state_mgr_check_legality() In-Reply-To: <486DD287.1050003@bull.net> References: <486C90BD.5040807@bull.net> <20080704064954.GF2319@sashak.voltaire.com> <486DD287.1050003@bull.net> Message-ID: <20080706114342.GC12450@sashak.voltaire.com> Hi Vincent, On 09:34 Fri 04 Jul , Vincent Ficet wrote: > > I have another question: I compiled opensm and ran it from a different > prefix. > Because I'm not root, I wanted opensm to log onto a path that was not > /var/cache. At first, I forgot to set the OSM_CACHE_DIR environment > variable and got the following error: > > Error from osm_opensm_init: IB_INSUFFICIENT_RESOURCES. > > Looking at strace, I quickly found what was wrong: > > lstat("/var/cache/opensm/", 0x7ffff7cbf970) = -1 ENOENT (No such file or > directory) > mkdir("/var/cache/opensm/", 0755) = -1 EACCES (Permission denied) > > Shouldn't opensm log something more accurate w.r.t this issue ? Strictly > speaking, it's not really an IB_INSUFFICIENT_RESOURCES issue. I guess this failure happened in osm_db_init() function (see in OpenSM log). This returns '1' on failure which is translated to IB_INSUFFICIENT_RESOURCES error code. Now in OpenSM most functions return status as 'ib_status_t' although not each such function (just as osm_db_init()) is IB related and not every error is IB related. I agree that this practice is not very successful in general. Sasha From sashak at voltaire.com Sun Jul 6 04:47:02 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 6 Jul 2008 14:47:02 +0300 Subject: [ofa-general] Re: [Infiniband-diags] Fix regenerate cache corner case. In-Reply-To: <1214863890.14097.73.camel@cardanus.llnl.gov> References: <1214863890.14097.73.camel@cardanus.llnl.gov> Message-ID: <20080706114702.GD12450@sashak.voltaire.com> On 15:11 Mon 30 Jun , Al Chu wrote: > Hey Sasha, > > Fixes a corner case I noticed. Nothing too fancy. > > Al > > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > From d1c5008c9cdc2a1c065b70d772fe11c527b05bcc Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Mon, 30 Jun 2008 15:09:49 -0700 > Subject: [PATCH] fix regenerate cache corner case > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From ogerlitz at voltaire.com Sun Jul 6 05:07:43 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 06 Jul 2008 15:07:43 +0300 Subject: [ofa-general] Re: loading/reading the value of qos_ca_high_limit In-Reply-To: <20080706112333.GA12450@sashak.voltaire.com> References: <20080706112333.GA12450@sashak.voltaire.com> Message-ID: <4870B58F.7030803@voltaire.com> Sasha Khapyorsky wrote: > It is encoded in PortInfo (VLHighLimit and OperVLs): OK, got it and also the second comment on the defaults. So... what is the practical difference between "operational VLs" and "max VLs" does one of them require to be burned through the HCA/Switch firmware where the other is configurable by the SM? Or. > > smpquery portinfo 3 > > # Port info: Lid 3 port 0 > VLCap:...........................VL0-7 > VLHighLimit:.....................0 > VLArbHighCap:....................8 > VLArbLowCap:.....................8 > VLStallCount:....................7 > OperVLs:.........................VL0-7 From sashak at voltaire.com Sun Jul 6 05:20:07 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 6 Jul 2008 15:20:07 +0300 Subject: [ofa-general] Re: loading/reading the value of qos_ca_high_limit In-Reply-To: <4870B58F.7030803@voltaire.com> References: <20080706112333.GA12450@sashak.voltaire.com> <4870B58F.7030803@voltaire.com> Message-ID: <20080706122007.GE12450@sashak.voltaire.com> On 15:07 Sun 06 Jul , Or Gerlitz wrote: > > So... what is the practical difference between "operational VLs" and "max > VLs" Guess you mean "operational VLs" against "VL Capability". VLCap is read only field, which indicates how many VLs port is capable to use. Sasha From vlad at mellanox.co.il Sun Jul 6 05:32:14 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Sun, 6 Jul 2008 15:32:14 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: fix bug in LRO Message-ID: <20080706123214.GA1666@mellanox.co.il> lro_flush_all() has to be called prior to calling netif_rx_complete. Failure to do this may cause ipoib_poll() to get called again while lro_flush_all() takes place and result in data corruption. Signed-off-by: Eli Cohen Signed-off-by: Vladimir Sokolovsky --- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index ca5afcd..c0fffe9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -443,6 +443,10 @@ poll_more: } if (done < budget) { + + if (dev->features & NETIF_F_LRO) + lro_flush_all(&priv->lro.lro_mgr); + netif_rx_complete(dev, napi); if (unlikely(ib_req_notify_cq(priv->recv_cq, IB_CQ_NEXT_COMP | @@ -451,9 +455,6 @@ poll_more: goto poll_more; } - if (dev->features & NETIF_F_LRO) - lro_flush_all(&priv->lro.lro_mgr); - return done; } -- 1.5.5.1 From pw at osc.edu Sun Jul 6 07:40:31 2008 From: pw at osc.edu (Pete Wyckoff) Date: Sun, 6 Jul 2008 10:40:31 -0400 Subject: [ofa-general] [PATCH] krping: free child_cm_id in all error cases Message-ID: <20080706144031.GA20054@osc.edu> Found a problem in krping while testing a broken iwarp device. Against git head. Not exactly tested, but works on my hacked-up older version of krping. -- Pete Anytime after the server has received a CONNECT_REQUEST, the child_cm_id exists and must be destroyed. Fix error paths to make sure this always happens, else the rdma_cm_id leaks and devices can not be unloaded. Signed-off-by: Pete Wyckoff --- krping.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/krping.c b/krping.c index d8fcdf2..4a6c382 100644 --- a/krping.c +++ b/krping.c @@ -1589,7 +1589,7 @@ static void krping_run_server(struct krping_cb *cb) ret = krping_setup_qp(cb, cb->child_cm_id); if (ret) { printk(KERN_ERR PFX "setup_qp failed: %d\n", ret); - return; + goto err0; } ret = krping_setup_buffers(cb); @@ -1619,11 +1619,12 @@ static void krping_run_server(struct krping_cb *cb) else krping_test_server(cb); rdma_disconnect(cb->child_cm_id); - rdma_destroy_id(cb->child_cm_id); err2: krping_free_buffers(cb); err1: krping_free_qp(cb); +err0: + rdma_destroy_id(cb->child_cm_id); } static void krping_test_client(struct krping_cb *cb) -- 1.5.5.1 From 88f9506a9a at ronblackagency.com Sun Jul 6 06:04:41 2008 From: 88f9506a9a at ronblackagency.com (dunn craddock) Date: Sun, 06 Jul 2008 13:04:41 +0000 Subject: [ofa-general] to general Message-ID: <000401c8df77$03684760$cfdbe190@fufppsed> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avalwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From tziporet at mellanox.co.il Sun Jul 6 08:01:22 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Sun, 6 Jul 2008 18:01:22 +0300 Subject: [ofa-general] List of libraries in OFED Message-ID: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> Hi All, This is the list of libraries in OFED We wish to define which are the libraries that are public and their API is being preserved from release to release. There are libraries I already classified, and there are some I am not sure about Each owner - please reply regarding the libraries that are under unknown Also - if I missed any library please add it to the list Thanks, Tziporet Public: ======= * libibcommon * libdat2 * libdat * libibcm * libibverbs * libibmad * libibumad * libsdp * librdmacm Private: ======== * libcxgb3 * libehca * libipathverbs * libmlx4 * libmthca * libnes Unknown: ======== * libibdmcom * libdaplcma * libdaplofa * libibdm * libibis * libibmscli * libopensm * libosmcomp * libosmvendor * libosmvendor_openib * libumad2sim From aj.guillon at gmail.com Sun Jul 6 11:02:21 2008 From: aj.guillon at gmail.com (Adrien Guillon) Date: Sun, 6 Jul 2008 14:02:21 -0400 Subject: [ofa-general] Getting Started with RDMA Message-ID: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> Hi everyone, I'm just getting started with RDMA... in particular my environment consists of Voltaire Infiniband equipment... although I would like to maintain platform independence as much as possible. I'm looking for a user space library that supports atomic operations, and remote memory transfers. I've looked a bit into librdmacm, basically looking at rdma_cm.h for some general information. I haven't been able to find much documentation on getting started with librdmacm, so links to docs would be greatly appreciated. Perhaps librdmacm isn't even the right library for me. My understanding is that atomic operation support is optional with Infiniband devices, however for my requirements it would be an absolute necessity that hardware supports it. My target platform is POSIX-compliant systems, in particular Linux. My goal is to create some C++ libraries for RDMA access, which can be used as a building block for other cluster applications. Thanks, AJ -------------- next part -------------- An HTML attachment was scrubbed... URL: From 099andrewkohler at email.uophx.edu Sun Jul 6 09:51:48 2008 From: 099andrewkohler at email.uophx.edu (granger ranjan) Date: Sun, 06 Jul 2008 16:51:48 +0000 Subject: [ofa-general] Best of Chloe, Dior, UGG Message-ID: <000701c8df97$035fb4e1$eb62ea85@byewdhl> The world's generous de luxe store for shoes and bags is just one click away. Recommended by thousands of satisfied customers all around the world, we carry many of populars brands including: Chloe Armani Coach D&G UGG Here you willc see millions of best designs for shoes, and leather products, at rock bottom pri'c'ing. Sale ends this weekend, so visit web site immediately and start pampering yourself and your loved second half. - Visit our site: www.shoequality[DOT]com (copy this link then replace "[DOT]" to ".") From olaf.kirch at oracle.com Sun Jul 6 12:38:28 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Sun, 6 Jul 2008 21:38:28 +0200 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807041346.52282.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041207.40490.olaf.kirch@oracle.com> <200807041346.52282.olaf.kirch@oracle.com> Message-ID: <200807062138.29103.olaf.kirch@oracle.com> On Friday 04 July 2008 13:46:51 Olaf Kirch wrote: > We have two choices here Okay, there's a third way to fix this, which may even work with today's Oracle IPC code. The hack is to not submit the fastreg WR until we post a SEND that transmits the rkey to a peer. Then - we have a QP to post the WRs to - we post the fastreg and the SEND to the same QP, ensuring that the peer doesn't initiate the RDMA before we've set up the memory registration. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From rdreier at cisco.com Sun Jul 6 14:31:20 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 14:31:20 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: fix bug in LRO In-Reply-To: <20080706123214.GA1666@mellanox.co.il> (Vladimir Sokolovsky's message of "Sun, 6 Jul 2008 15:32:14 +0300") References: <20080706123214.GA1666@mellanox.co.il> Message-ID: thanks, I rolled this into the main LRO patch. From rdreier at cisco.com Sun Jul 6 14:41:39 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 14:41:39 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: remove unused code In-Reply-To: <20080706082535.GA11165@mtls03> (Eli Cohen's message of "Sun, 6 Jul 2008 11:25:35 +0300") References: <20080706082535.GA11165@mtls03> Message-ID: nice cleanup, applied. From rdreier at cisco.com Sun Jul 6 14:44:44 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 14:44:44 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: remove priv->mcast_mutex In-Reply-To: <20080706082617.GA11207@mtls03> (Eli Cohen's message of "Sun, 6 Jul 2008 11:26:17 +0300") References: <20080706082617.GA11207@mtls03> Message-ID: thanks, applied.. I think this mutex is a relic of the days of THCA and shaky locking there... From rdreier at cisco.com Sun Jul 6 14:49:58 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 14:49:58 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: set QP qkey once after joining broadcast group In-Reply-To: <20080706082843.GA11244@mtls03> (Eli Cohen's message of "Sun, 6 Jul 2008 11:28:43 +0300") References: <20080706082843.GA11244@mtls03> Message-ID: thanks, applied. I assume the reason for this is just that setting the Q_Key all the time is a waste -- or is there some deeper reason here? From rdreier at cisco.com Sun Jul 6 14:55:32 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 14:55:32 -0700 Subject: [ofa-general] Re: [ewg] Re: [PATCH] IB/ipoib: set QP qkey once after joining broadcast group In-Reply-To: (Roland Dreier's message of "Sun, 06 Jul 2008 14:49:58 -0700") References: <20080706082843.GA11244@mtls03> Message-ID: and here's one more cleanup I just comitted, inspired by this work: commit b99598319c9e3501bc75f3e82d0c145793b727c1 Author: Roland Dreier Date: Sun Jul 6 14:55:18 2008 -0700 IPoIB: Get rid of ipoib_mcast_detach() wrapper ipoib_mcast_detach() does nothing except call ib_detach_mcast(), so just use the core API in the one place that does a multicast group detach. add/remove: 0/1 grow/shrink: 0/1 up/down: 0/-105 (-105) function old new delta ipoib_mcast_leave 357 319 -38 ipoib_mcast_detach 67 - -67 Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index dc10449..5f3c6cd 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -485,8 +485,6 @@ void ipoib_path_iter_read(struct ipoib_path_iter *iter, int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey); -int ipoib_mcast_detach(struct net_device *dev, u16 mlid, - union ib_gid *mgid); int ipoib_init_qp(struct net_device *dev); int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 55ebd95..71add7a 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -627,10 +627,10 @@ static int ipoib_mcast_leave(struct net_device *dev, struct ipoib_mcast *mcast) IPOIB_GID_ARG(mcast->mcmember.mgid)); /* Remove ourselves from the multicast group */ - ret = ipoib_mcast_detach(dev, be16_to_cpu(mcast->mcmember.mlid), - &mcast->mcmember.mgid); + ret = ib_detach_mcast(priv->qp, &mcast->mcmember.mgid, + be16_to_cpu(mcast->mcmember.mlid)); if (ret) - ipoib_warn(priv, "ipoib_mcast_detach failed (result = %d)\n", ret); + ipoib_warn(priv, "ib_detach_mcast failed (result = %d)\n", ret); } return 0; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index ba7c886..6832511 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -72,18 +72,6 @@ out: return ret; } -int ipoib_mcast_detach(struct net_device *dev, u16 mlid, union ib_gid *mgid) -{ - struct ipoib_dev_priv *priv = netdev_priv(dev); - int ret; - - ret = ib_detach_mcast(priv->qp, mgid, mlid); - if (ret) - ipoib_warn(priv, "ib_detach_mcast failed (result = %d)\n", ret); - - return ret; -} - int ipoib_init_qp(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); From rdreier at cisco.com Sun Jul 6 14:59:36 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 14:59:36 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: Use rtnl lock/unlock when changing device flags In-Reply-To: <20080703085210.GA8511@mtls03> (Eli Cohen's message of "Thu, 3 Jul 2008 11:52:10 +0300") References: <20080703085210.GA8511@mtls03> Message-ID: thanks, applied. From rdreier at cisco.com Sun Jul 6 15:02:02 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 15:02:02 -0700 Subject: [ofa-general] Re: [PATCH] Subject: [PATCH] IB/IPoIB: Fix change of mtu to use kernel API In-Reply-To: <20080703085245.GA8551@mtls03> (Eli Cohen's message of "Thu, 3 Jul 2008 11:52:45 +0300") References: <20080703085245.GA8551@mtls03> Message-ID: thanks, applied From rdreier at cisco.com Sun Jul 6 15:04:25 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 15:04:25 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipath - Change the vendor_id reported by ibv_query_device() In-Reply-To: <20080703230207.466.91454.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Thu, 03 Jul 2008 16:02:07 -0700") References: <20080703230207.466.91454.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied From sashak at voltaire.com Sun Jul 6 17:01:49 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 7 Jul 2008 03:01:49 +0300 Subject: [ofa-general] [PATCH] management: revert configure.in update configure.in on 'make dist' Message-ID: <20080707000149.GA15010@sashak.voltaire.com> Recently configure.in automatic update was added. This setup there a package version in accordance with tree state during 'make dist'. However this method has a disadvantage - distributed configure.in template is newer than an actual './configure' script, it results to an additional (automatic) reconfiguration during build time. This patch removes this "smart" configure.in update. Instead we will handle versions directly in a daily build script. Signed-off-by: Sasha Khapyorsky --- infiniband-diags/Makefile.am | 4 ---- libibcommon/Makefile.am | 4 ---- libibmad/Makefile.am | 4 ---- libibumad/Makefile.am | 4 ---- opensm/Makefile.am | 4 ---- 5 files changed, 0 insertions(+), 20 deletions(-) diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index b6228b5..c22ba5e 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -119,10 +119,6 @@ dist-hook: if [ -x $(top_srcdir)/../gen_chlog.sh ] ; then \ $(top_srcdir)/../gen_chlog.sh $(PACKAGE) > $(distdir)/ChangeLog ; \ fi - if [ -x $(top_srcdir)/../gen_ver.sh ] ; then \ - ver=`$(top_srcdir)/../gen_ver.sh $(PACKAGE)` ; \ - sed -e '/AC_INIT/s/$(PACKAGE), .*,/$(PACKAGE), '$$ver',/' $(top_srcdir)/configure.in > $(distdir)/configure.in ; \ - fi # install this to a default location. install-data-hook: diff --git a/libibcommon/Makefile.am b/libibcommon/Makefile.am index dd8e264..75889f4 100644 --- a/libibcommon/Makefile.am +++ b/libibcommon/Makefile.am @@ -30,7 +30,3 @@ dist-hook: if [ -x $(top_srcdir)/../gen_chlog.sh ] ; then \ $(top_srcdir)/../gen_chlog.sh $(PACKAGE) > $(distdir)/ChangeLog ; \ fi - if [ -x $(top_srcdir)/../gen_ver.sh ] ; then \ - ver=`$(top_srcdir)/../gen_ver.sh $(PACKAGE)` ; \ - sed -e '/AC_INIT/s/$(PACKAGE), .*,/$(PACKAGE), '$$ver',/' $(top_srcdir)/configure.in > $(distdir)/configure.in ; \ - fi diff --git a/libibmad/Makefile.am b/libibmad/Makefile.am index 9def2fe..beae1a4 100644 --- a/libibmad/Makefile.am +++ b/libibmad/Makefile.am @@ -32,7 +32,3 @@ dist-hook: if [ -x $(top_srcdir)/../gen_chlog.sh ] ; then \ $(top_srcdir)/../gen_chlog.sh $(PACKAGE) > $(distdir)/ChangeLog ; \ fi - if [ -x $(top_srcdir)/../gen_ver.sh ] ; then \ - ver=`$(top_srcdir)/../gen_ver.sh $(PACKAGE)` ; \ - sed -e '/AC_INIT/s/$(PACKAGE), .*,/$(PACKAGE), '$$ver',/' $(top_srcdir)/configure.in > $(distdir)/configure.in ; \ - fi diff --git a/libibumad/Makefile.am b/libibumad/Makefile.am index f014f57..1e3e6fd 100644 --- a/libibumad/Makefile.am +++ b/libibumad/Makefile.am @@ -45,10 +45,6 @@ dist-hook: if [ -x $(top_srcdir)/../gen_chlog.sh ] ; then \ $(top_srcdir)/../gen_chlog.sh $(PACKAGE) > $(distdir)/ChangeLog ; \ fi - if [ -x $(top_srcdir)/../gen_ver.sh ] ; then \ - ver=`$(top_srcdir)/../gen_ver.sh $(PACKAGE)` ; \ - sed -e '/AC_INIT/s/$(PACKAGE), .*,/$(PACKAGE), '$$ver',/' $(top_srcdir)/configure.in > $(distdir)/configure.in ; \ - fi install-data-hook: cd $(DESTDIR)$(mandir)/man3 && \ diff --git a/opensm/Makefile.am b/opensm/Makefile.am index fca5319..0fb1363 100644 --- a/opensm/Makefile.am +++ b/opensm/Makefile.am @@ -29,7 +29,3 @@ dist-hook: $(EXTRA_DIST) if [ -x $(top_srcdir)/../gen_chlog.sh ] ; then \ $(top_srcdir)/../gen_chlog.sh $(PACKAGE) > $(distdir)/ChangeLog ; \ fi - if [ -x $(top_srcdir)/../gen_ver.sh ] ; then \ - ver=`$(top_srcdir)/../gen_ver.sh $(PACKAGE)` ; \ - sed -e '/AC_INIT/s/$(PACKAGE), .*,/$(PACKAGE), '$$ver',/' $(top_srcdir)/configure.in > $(distdir)/configure.in ; \ - fi -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Sun Jul 6 17:02:46 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 7 Jul 2008 03:02:46 +0300 Subject: [ofa-general] [PATCH] management/make.dist: daily build updates In-Reply-To: <20080707000149.GA15010@sashak.voltaire.com> References: <20080707000149.GA15010@sashak.voltaire.com> Message-ID: <20080707000246.GB15010@sashak.voltaire.com> Daily versions will be generated in format: ${package}-${base_version}-${date}-${git_abbreviation} For example: opensm-3.2.1-20080707-12af490 Tarball files will have a same names but with .tar.gz suffix. Daily build will generate new tarball only when actual changes have been made. Signed-off-by: Sasha Khapyorsky --- make.dist | 26 ++++++++++++++++++++++++-- 1 files changed, 24 insertions(+), 2 deletions(-) diff --git a/make.dist b/make.dist index 29cee0c..2e67940 100755 --- a/make.dist +++ b/make.dist @@ -51,6 +51,12 @@ if [ -z "$1" ]; then usage; exit 1; fi if [ "$1" != "daily" -a "$1" != "release" ]; then usage; exit 1; fi +git diff-index --quiet HEAD -- $package > /dev/null 2>&1 +if [ $? -eq 1 ] ; then + echo "git tree is dirty, please commit or undo the changes" + #exit 2 +fi + if [ -z "$TARGETS" ]; then TARGETS="libibcommon libibumad libibmad opensm infiniband-diags" fi @@ -95,7 +101,6 @@ for target in $TARGETS; do # incrementally higher than the last officially released tarball. RELEASE=1 echo $RELEASE > $TMPDIR/$target.release - TARBALL=$target-$VERSION.tar.gz if [ ! -z "$2" ]; then if [ $2 = "signed" ]; then @@ -108,6 +113,16 @@ for target in $TARGETS; do fi elif [ "$1" = "daily" ]; then DATE=`date +%Y%m%d` + git_rev=`./gen_ver.sh $target | sed -e 's/^'$VERSION'-//'` + VERSION=$VERSION-$DATE-$git_rev + if [ -f $TMPDIR/$target.gitrev ]; then + old_rev=`cat $TMPDIR/$target.gitrev` + fi + echo $git_rev > $TMPDIR/$target.gitrev + if [ "$old_rev" = "$git_rev" ] ; then + echo "No daily build is needed for '$target' ($git_rev)" + continue + fi if [ -f $TMPDIR/$target.release ]; then RELEASE=`cat $TMPDIR/$target.release` RELEASE=`expr $RELEASE + 1` @@ -116,12 +131,19 @@ for target in $TARGETS; do fi echo $RELEASE > $TMPDIR/$target.release RELEASE=0.${RELEASE}.${DATE}git - TARBALL=$target-$VERSION-$RELEASE.tar.gz + cat $target/configure.in \ + | sed -e '/AC_INIT/s/'$target', .*,/'$target', '$VERSION',/' \ + > configure.in.new + diff $target/configure.in configure.in.new > /dev/null \ + || cat configure.in.new > $target/configure.in fi + TARBALL=$target-$VERSION.tar.gz + echo "Creating $TMPDIR/$TARBALL" ( cd $target && ./autogen.sh && RELEASE=$RELEASE TARBALL=$TARBALL ./configure && make dist && mv $target-$VERSION.tar.gz $TMPDIR/$TARBALL ) || exit $? + git checkout $target/configure.in done -- 1.5.5.1.178.g1f811 From rdreier at cisco.com Sun Jul 6 21:49:26 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 21:49:26 -0700 Subject: [ofa-general] Re: [PATCH 1/4] rdma/addr: keep pointer to the netdevice in struct rdma_dev_addr In-Reply-To: (Or Gerlitz's message of "Thu, 19 Jun 2008 17:06:11 +0300 (IDT)") References: Message-ID: > Keep a pointer to the local (src) netdevice in struct rdma_dev_addr. In a > High-Availability scheme this information can be used by the rdma-cm to align RDMA > sessions to use the same links as the IP stack does under fail-over and route change cases. I don't understand how keeping the netdevice is related to this chunk: > --- infiniband.orig/drivers/infiniband/core/cma.c > +++ infiniband/drivers/infiniband/core/cma.c > @@ -1002,6 +1002,7 @@ static struct rdma_id_private *cma_new_c > union cma_ip_addr *src, *dst; > __be16 port; > u8 ip_ver; > + int ret; > > if (cma_get_net_info(ib_event->private_data, listen_id->ps, > &ip_ver, &port, &src, &dst)) > @@ -1026,10 +1027,11 @@ static struct rdma_id_private *cma_new_c > if (rt->num_paths == 2) > rt->path_rec[1] = *ib_event->param.req_rcvd.alternate_path; > > - ib_addr_set_sgid(&rt->addr.dev_addr, &rt->path_rec[0].sgid); > ib_addr_set_dgid(&rt->addr.dev_addr, &rt->path_rec[0].dgid); > - ib_addr_set_pkey(&rt->addr.dev_addr, be16_to_cpu(rt->path_rec[0].pkey)); > - rt->addr.dev_addr.dev_type = RDMA_NODE_IB_CA; > + ret = rdma_translate_ip(&id->route.addr.src_addr, > + &id->route.addr.dev_addr); > + if (ret) > + goto destroy_id; > > id_priv = container_of(id, struct rdma_id_private, id); > id_priv->state = CMA_CONNECT; From rdreier at cisco.com Sun Jul 6 22:08:56 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 22:08:56 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: reduce CM tx object size In-Reply-To: <1211731181.13769.46.camel@mtls03> (Eli Cohen's message of "Sun, 25 May 2008 18:59:41 +0300") References: <1211731181.13769.46.camel@mtls03> Message-ID: thanks, applied From rdreier at cisco.com Sun Jul 6 22:15:03 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 22:15:03 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: increase ring sizes In-Reply-To: <1211786440.13769.54.camel@mtls03> (Eli Cohen's message of "Mon, 26 May 2008 10:20:40 +0300") References: <1211786440.13769.54.camel@mtls03> Message-ID: thanks, applied. From ogerlitz at voltaire.com Sun Jul 6 22:37:33 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 07 Jul 2008 08:37:33 +0300 Subject: [ofa-general] Re: [PATCH] IB/ipoib: remove unused code In-Reply-To: References: <20080706082535.GA11165@mtls03> Message-ID: <4871AB9D.3010002@voltaire.com> Roland Dreier wrote: > nice cleanup, applied. Yes, eventually we could clean it up with commit b3e2749bf32f61e7beb259eb7cfb066d2ec6ad65 "IPoIB: Don't drop multicast sends when they can be queued", but, anyway good to have it sorted out now. Or. From ogerlitz at voltaire.com Sun Jul 6 22:54:19 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 07 Jul 2008 08:54:19 +0300 Subject: [ofa-general] Re: [PATCH 1/4] rdma/addr: keep pointer to the netdevice in struct rdma_dev_addr In-Reply-To: References: Message-ID: <4871AF8B.7020207@voltaire.com> Roland Dreier wrote: > > Keep a pointer to the local (src) netdevice in struct rdma_dev_addr. In a > > High-Availability scheme this information can be used by the rdma-cm to align RDMA > > sessions to use the same links as the IP stack does under fail-over and route change cases. > > I don't understand how keeping the netdevice is related to this chunk: The rdma-cm doesn't issue address resolution (route lookup, neigh lookup, etc) in the passive side of the connection and hence there is some asymmetry between the code path taken at the client side vs the server side. This difference had two implications: - the passive side network device was not resolved, where now it is resolved by rdma_translate_ip - there was some trivial code duplication between rdma_copy_addr to cma_new_conn_id, which is now removed as with this patch the chain of calls becomes cma_new_conn_id --> rdma_translate_ip --> rdma_copy_addr Or From rdreier at cisco.com Sun Jul 6 22:59:28 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 22:59:28 -0700 Subject: [ofa-general] [PATCH] IB/mthca: Remove "stop" flag for catastrophic error polling timer Message-ID: Since we use del_timer_sync() anyway, there's no need for an additional flag to tell the timer not to rearm. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mthca/mthca_catas.c | 15 ++------------- drivers/infiniband/hw/mthca/mthca_dev.h | 1 - 2 files changed, 2 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_catas.c b/drivers/infiniband/hw/mthca/mthca_catas.c index 40573e4..50e2792 100644 --- a/drivers/infiniband/hw/mthca/mthca_catas.c +++ b/drivers/infiniband/hw/mthca/mthca_catas.c @@ -126,7 +126,6 @@ static void handle_catas(struct mthca_dev *dev) static void poll_catas(unsigned long dev_ptr) { struct mthca_dev *dev = (struct mthca_dev *) dev_ptr; - unsigned long flags; int i; for (i = 0; i < dev->catas_err.size; ++i) @@ -135,13 +134,8 @@ static void poll_catas(unsigned long dev_ptr) return; } - spin_lock_irqsave(&catas_lock, flags); - if (!dev->catas_err.stop) - mod_timer(&dev->catas_err.timer, - jiffies + MTHCA_CATAS_POLL_INTERVAL); - spin_unlock_irqrestore(&catas_lock, flags); - - return; + mod_timer(&dev->catas_err.timer, + jiffies + MTHCA_CATAS_POLL_INTERVAL); } void mthca_start_catas_poll(struct mthca_dev *dev) @@ -149,7 +143,6 @@ void mthca_start_catas_poll(struct mthca_dev *dev) unsigned long addr; init_timer(&dev->catas_err.timer); - dev->catas_err.stop = 0; dev->catas_err.map = NULL; addr = pci_resource_start(dev->pdev, 0) + @@ -180,10 +173,6 @@ void mthca_start_catas_poll(struct mthca_dev *dev) void mthca_stop_catas_poll(struct mthca_dev *dev) { - spin_lock_irq(&catas_lock); - dev->catas_err.stop = 1; - spin_unlock_irq(&catas_lock); - del_timer_sync(&dev->catas_err.timer); if (dev->catas_err.map) { diff --git a/drivers/infiniband/hw/mthca/mthca_dev.h b/drivers/infiniband/hw/mthca/mthca_dev.h index 2997d8d..ee4d073 100644 --- a/drivers/infiniband/hw/mthca/mthca_dev.h +++ b/drivers/infiniband/hw/mthca/mthca_dev.h @@ -277,7 +277,6 @@ struct mthca_mcg_table { struct mthca_catas_err { u64 addr; u32 __iomem *map; - unsigned long stop; u32 size; struct timer_list timer; struct list_head list; -- 1.5.6 From rdreier at cisco.com Sun Jul 6 23:01:10 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 23:01:10 -0700 Subject: [ofa-general] [PATCH] IB/mthca: Use round_jiffies() for catastrophic error polling timer Message-ID: Exactly when the catastrophic error polling timer function runs is not important, so use round_jiffies() to save unnecessary wakeups. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mthca/mthca_catas.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_catas.c b/drivers/infiniband/hw/mthca/mthca_catas.c index 50e2792..cc440f9 100644 --- a/drivers/infiniband/hw/mthca/mthca_catas.c +++ b/drivers/infiniband/hw/mthca/mthca_catas.c @@ -135,7 +135,7 @@ static void poll_catas(unsigned long dev_ptr) } mod_timer(&dev->catas_err.timer, - jiffies + MTHCA_CATAS_POLL_INTERVAL); + round_jiffies(jiffies + MTHCA_CATAS_POLL_INTERVAL)); } void mthca_start_catas_poll(struct mthca_dev *dev) -- 1.5.6 From rdreier at cisco.com Sun Jul 6 23:19:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 06 Jul 2008 23:19:05 -0700 Subject: [ofa-general] [PATCH] IB/mthca: Fix check of max_send_sge for special QPs Message-ID: The MLX transport requires two extra gather entries for sends (one for the header and one for the checksum at the end, as the comment says). However the code checked that max_recv_sge was not too big, instead of checking max_send_sge as it should have. Fix the code to check the correct condition. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mthca/mthca_qp.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c index 77b52b3..f5081bf 100644 --- a/drivers/infiniband/hw/mthca/mthca_qp.c +++ b/drivers/infiniband/hw/mthca/mthca_qp.c @@ -1249,10 +1249,10 @@ static int mthca_set_qp_size(struct mthca_dev *dev, struct ib_qp_cap *cap, return -EINVAL; /* - * For MLX transport we need 2 extra S/G entries: + * For MLX transport we need 2 extra send gather entries: * one for the header and one for the checksum at the end */ - if (qp->transport == MLX && cap->max_recv_sge + 2 > dev->limits.max_sg) + if (qp->transport == MLX && cap->max_send_sge + 2 > dev->limits.max_sg) return -EINVAL; if (mthca_is_memfree(dev)) { -- 1.5.6 From ogerlitz at voltaire.com Sun Jul 6 23:23:54 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 07 Jul 2008 09:23:54 +0300 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <200807061300.31811.jackm@dev.mellanox.co.il> References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> <200807061300.31811.jackm@dev.mellanox.co.il> Message-ID: <4871B67A.6050704@voltaire.com> Jack Morgenstein wrote: > In the case multicast loopback, a creation flag is most appropriate, because this is really a qp-creation-only issue (the kernel core layer only needs this info at QP creation time). > > Probably, the least short-term pain (no ABI change) would be to use the last reserved create-qp ABI byte for flags, while defining the flags field in ibv_qp_init_attr as uint32_t (preparing for more than 8 future flags). Hi Jack, Thanks a lot for the detailed feedback, which is very helpful for this work Or. From eli at dev.mellanox.co.il Sun Jul 6 23:40:11 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 7 Jul 2008 09:40:11 +0300 Subject: [ofa-general] Re: [ewg] Re: [PATCH] IB/ipoib: set QP qkey once after joining broadcast group In-Reply-To: References: <20080706082843.GA11244@mtls03> Message-ID: <20080707064011.GA29673@mtls03> On Sun, Jul 06, 2008 at 02:49:58PM -0700, Roland Dreier wrote: > thanks, applied. I assume the reason for this is just that setting the > Q_Key all the time is a waste -- or is there some deeper reason here? Nothing deeper. Just save the unnecessary modify QP invocations. From eli at dev.mellanox.co.il Mon Jul 7 00:32:56 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 7 Jul 2008 10:32:56 +0300 Subject: [ofa-general] Re: [ewg] [PATCH] Subject: [PATCH] IB/ipoib: set max CM MTU when moving to CM mode In-Reply-To: <20080703085329.GA8583@mtls03> References: <20080703085329.GA8583@mtls03> Message-ID: <20080707073256.GB29673@mtls03> On Thu, Jul 03, 2008 at 11:53:29AM +0300, Eli Cohen wrote: Roland, what's your opinion about this patch? > This will relieve the user from the need to restore CM mode MTU > every time he switchs from UD to CM mode. With the current code, > if the user fails to reset the MTU to a decent higher value, he > will get poor performance. > > Signed-off-by: Eli Cohen > --- > --- > drivers/infiniband/ulp/ipoib/ipoib_cm.c | 4 ++++ > 1 files changed, 4 insertions(+), 0 deletions(-) > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > index 99a4daf..eba92e7 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > @@ -1384,8 +1384,12 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, > ipoib_warn(priv, "enabling connected mode " > "will cause multicast packet drops\n"); > > + if (ipoib_cm_max_mtu(dev) > priv->mcast_mtu) > + ipoib_warn(priv, "mtu > %d will cause multicast packet drops.\n", > + priv->mcast_mtu); > rtnl_lock(); > dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); > + dev_set_mtu(dev, ipoib_cm_max_mtu(dev)); > rtnl_unlock(); > priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; > > -- > 1.5.6 > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg From ogerlitz at voltaire.com Mon Jul 7 01:20:17 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 7 Jul 2008 11:20:17 +0300 (IDT) Subject: [ofa-general] Re: performance drop for datagram mode with the new connectx FW In-Reply-To: References: Message-ID: > manage narrowing this a little further, the issue seems most > notable under long messages sent over datagram mode, could be > an issue in the LSO engine ?! see in this table, > > ==================================================== > mode mtu size bw note > ==================================================== > datagram 2044 64000 30 <---------- problem > # netperf -H 10.10.0.90 -fM -l 20 -D 1, -- -m 64000 OK, it turns out that this specific parameters set to netperf / TCP_STREAM was the source for the performance drop I saw, setting also the socket send buffer changes everything (all tests use datagram mode, mtu 2044, msg size 64000) ==================================================== send_buf recv_buf BW (MB/s) ==================================================== NA NA 30 32000 NA 640 64000 NA 750 128000 NA 790 128000 128000 840 On this system / current settings, I get quite the same numbers for both firmware version (2.3 and 2.5) Or. # netperf -H 10.10.0.90 -fM -l 20 -D 1, -- -m 64000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 44.11 MBytes/s over 10.68 seconds Interim result: 34.25 MBytes/s over 2.49 seconds Interim result: 67.30 MBytes/s over 1.01 seconds Interim result: 67.28 MBytes/s over 1.00 seconds Interim result: 66.72 MBytes/s over 1.01 seconds Interim result: 33.31 MBytes/s over 2.00 seconds Interim result: 22.63 MBytes/s over 1.47 seconds Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. MBytes/sec 87380 16384 64000 20.66 42.95 # netperf -H 10.10.0.90 -fM -l 20 -D 1, -- -m 64000 -s 32000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 643.92 MBytes/s over 1.00 seconds Interim result: 646.37 MBytes/s over 1.00 seconds Interim result: 644.34 MBytes/s over 1.00 seconds Interim result: 644.39 MBytes/s over 1.00 seconds Interim result: 643.47 MBytes/s over 1.00 seconds Interim result: 643.22 MBytes/s over 1.00 seconds Interim result: 643.55 MBytes/s over 1.00 seconds Interim result: 642.41 MBytes/s over 1.00 seconds Interim result: 640.98 MBytes/s over 1.00 seconds Interim result: 641.88 MBytes/s over 1.00 seconds Interim result: 642.67 MBytes/s over 1.00 seconds Interim result: 642.45 MBytes/s over 1.00 seconds Interim result: 642.25 MBytes/s over 1.00 seconds Interim result: 642.23 MBytes/s over 1.00 seconds Interim result: 640.45 MBytes/s over 1.00 seconds Interim result: 641.84 MBytes/s over 1.00 seconds Interim result: 640.92 MBytes/s over 1.00 seconds Interim result: 639.94 MBytes/s over 1.00 seconds Interim result: 639.54 MBytes/s over 1.00 seconds Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. MBytes/sec 87380 64000 64000 20.00 642.22 [root at cto-gen2-1 ~]# netperf -H 10.10.0.90 -fM -l 20 -D 1, -- -m 64000 -s 64000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 753.91 MBytes/s over 1.00 seconds Interim result: 754.28 MBytes/s over 1.00 seconds Interim result: 752.74 MBytes/s over 1.00 seconds Interim result: 747.04 MBytes/s over 1.01 seconds Interim result: 747.66 MBytes/s over 1.00 seconds Interim result: 746.79 MBytes/s over 1.00 seconds Interim result: 749.77 MBytes/s over 1.00 seconds Interim result: 751.40 MBytes/s over 1.00 seconds Interim result: 748.91 MBytes/s over 1.00 seconds Interim result: 750.36 MBytes/s over 1.00 seconds Interim result: 747.39 MBytes/s over 1.00 seconds Interim result: 749.42 MBytes/s over 1.00 seconds Interim result: 750.04 MBytes/s over 1.00 seconds Interim result: 751.30 MBytes/s over 1.00 seconds Interim result: 750.37 MBytes/s over 1.00 seconds Interim result: 749.90 MBytes/s over 1.00 seconds Interim result: 747.94 MBytes/s over 1.00 seconds Interim result: 747.81 MBytes/s over 1.00 seconds Interim result: 744.92 MBytes/s over 1.00 seconds Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. MBytes/sec 87380 128000 64000 20.00 749.34 [root at cto-gen2-1 ~]# netperf -H 10.10.0.90 -fM -l 20 -D 1, -- -m 64000 -s 128000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 774.11 MBytes/s over 1.00 seconds Interim result: 786.26 MBytes/s over 1.00 seconds Interim result: 795.26 MBytes/s over 1.00 seconds Interim result: 795.31 MBytes/s over 1.00 seconds Interim result: 791.34 MBytes/s over 1.00 seconds Interim result: 792.93 MBytes/s over 1.00 seconds Interim result: 793.18 MBytes/s over 1.00 seconds Interim result: 792.43 MBytes/s over 1.00 seconds Interim result: 794.03 MBytes/s over 1.00 seconds Interim result: 791.21 MBytes/s over 1.00 seconds Interim result: 791.28 MBytes/s over 1.00 seconds Interim result: 790.57 MBytes/s over 1.00 seconds Interim result: 791.86 MBytes/s over 1.00 seconds Interim result: 792.43 MBytes/s over 1.00 seconds Interim result: 792.40 MBytes/s over 1.00 seconds Interim result: 793.32 MBytes/s over 1.00 seconds Interim result: 791.92 MBytes/s over 1.00 seconds Interim result: 793.45 MBytes/s over 1.00 seconds Interim result: 792.25 MBytes/s over 1.00 seconds Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. MBytes/sec 87380 256000 64000 20.00 791.32 # netperf -H 10.10.0.90 -fM -l 20 -D 1, -- -m 64000 -s 128000 -S 128000 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.10.0.90 (10.10.0.90) port 0 AF_INET : demo Interim result: 842.25 MBytes/s over 1.00 seconds Interim result: 845.21 MBytes/s over 1.00 seconds Interim result: 845.46 MBytes/s over 1.00 seconds Interim result: 844.63 MBytes/s over 1.00 seconds Interim result: 844.32 MBytes/s over 1.00 seconds Interim result: 842.70 MBytes/s over 1.00 seconds Interim result: 845.30 MBytes/s over 1.00 seconds Interim result: 843.44 MBytes/s over 1.00 seconds Interim result: 844.67 MBytes/s over 1.00 seconds Interim result: 843.80 MBytes/s over 1.00 seconds Interim result: 842.33 MBytes/s over 1.00 seconds Interim result: 842.04 MBytes/s over 1.00 seconds Interim result: 843.81 MBytes/s over 1.00 seconds Interim result: 841.76 MBytes/s over 1.00 seconds Interim result: 845.92 MBytes/s over 1.00 seconds Interim result: 843.63 MBytes/s over 1.00 seconds Interim result: 840.97 MBytes/s over 1.00 seconds Interim result: 841.30 MBytes/s over 1.00 seconds Interim result: 842.38 MBytes/s over 1.00 seconds Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. MBytes/sec 256000 256000 64000 20.00 843.31 From keshetti.mahesh at gmail.com Mon Jul 7 02:15:44 2008 From: keshetti.mahesh at gmail.com (Keshetti Mahesh) Date: Mon, 7 Jul 2008 14:45:44 +0530 Subject: [ofa-general] ibdmchk: how to consider multiple VLs while checking for credit loops ? Message-ID: <829ded920807070215g5e9ef6c4xf12082a69c09f93e@mail.gmail.com> > > > > LASH resolves credit loops by using different VLs, I don't think ibdmchk > > > > takes this into account, but don't know for sure. > > > > Yes, I have verified in ibdmchk that it considers only one VL while > > checking for > > credit loops. > > > > > I also think ibdmchk needs some support to handle LASH. I don't think it > > > is currently supported by it (although that is not documented AFAIK). > > > > > > > Is anyone currently working on this part (adding support to ibdmchk to > > handle LASH) > > in OFED community. After the last message in the above thread (which is way back), I didn't see anyone working on updating 'ibdmchk' to consider multiple VLs while checking the network for credit loops. 'ibdmchk' requires "path<->VL" (not SL2VL) association information in order to consider multIple VLs. But this information as of now is not dumped by OpenSM. A proper format should be defined to dump this information by OpenSM as this information is not straightforward like switch forwarding tables. Can anyone suggest me the best way to do it ? -Mahesh From vlad at lists.openfabrics.org Mon Jul 7 02:43:06 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Mon, 7 Jul 2008 02:43:06 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080707-0200 daily build status Message-ID: <20080707094306.505B1E6080F@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080707-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ponerology at mypinnacletech.com Mon Jul 7 03:31:10 2008 From: ponerology at mypinnacletech.com (Leif Watson) Date: Mon, 07 Jul 2008 19:31:10 +0900 Subject: [ofa-general] _EncoreDVD CS 3_ Message-ID: <000401c8e01b$e02f6380$0100007f@scupor> _ Adobe CS3 Master Collection for PC or MAC includes: _ InDesign CS3 _ Photoshop CS3 _ Illustrator CS3 _ Acrobat 8 Professional _ Flash CS3 Professional _ Dreamweaver CS3 _ Fireworks CS3 _ Contribute CS3 _ After Effects CS3 Professional _ Premiere Pro CS3 _ Encore DVD CS3 _ Soundbooth CS3 _ esoftfast . com in Internet Explorer _ System Requirements _ For PC: _ Intel Pentium 4 (1.4GHz processor for DV; 3.4GHz processor for HDV), Intel Centrino, Intel Xeon, (dual 2.8GHz processors for HD), or Intel Core _ Duo (or compatible) processor; SSE2-enabled processor required for AMD systems _ Microsoft Windows XP with Service Pack 2 or Microsoft Windows Vista Home Premium, Business, Ultimate, or Enterprise (certified for 32-bit editions) _ 1GB of RAM for DV; 2GB of RAM for HDV and HD; more RAM recommended when running multiple components _ 38GB of available hard-disk space (additional free space required during installation) _ Dedicated 7,200 RPM hard drive for DV and HDV editing; striped disk array storage (RAID 0) for HD; SCSI disk subsystem preferred _ Microsoft DirectX compatible sound card (multichannel ASIO-compatible sound card recommended) _ 1,280x1,024 monitor resolution with 32-bit color adapter _ DVD-ROM drive _ For MAC: _ PowerPC G4 or G5 or multicore Intel processor (Adobe Premiere Pro, Encore, and Soundbooth require a multicore Intel processor; Adobe OnLocation CS3 is a Windows application and may be used with Boot Camp) _ Mac OS X v.10.4.9; Java Runtime Environment 1.5 required for Adobe Version Cue CS3 Server _ 1GB of RAM for DV; 2GB of RAM for HDV and HD; more RAM recommended when running multiple components _ 36GB of available hard-disk space (additional free space required during installation) _ Dedicated 7,200 RPM hard drive for DV and HDV editing; striped disk array storage (RAID 0) for HD; SCSI disk subsystem preferred _ Core Audio compatible sound card _ 1,280x1,024 monitor resolution with 32-bit color adapter _ DVD-ROM drive_ DVD+-R burner required for DVD creation Clarke: Negotiate With Pakistan To Staunch Al-Qaida Sunni and Shiite leaders will meet in Iraq with Martin McGuinne, the deputy First Minister of Northern Ireland and former IRA leader, and Cyril Ramaphosa, the South African negotiator who helped end apartheid. They have been meeting privately with Iraq's politicians for two years to try and hammer out an agreement on reconciliation. From 3zsb at cjlu.edu.cn Mon Jul 7 02:17:06 2008 From: 3zsb at cjlu.edu.cn (inglebert thu) Date: Mon, 07 Jul 2008 09:17:06 +0000 Subject: [ofa-general] to general Message-ID: <000501c8e021$05e4d62c$4740b08f@rdgsq> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avawewosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From 19calford at mjhs.org Mon Jul 7 02:16:40 2008 From: 19calford at mjhs.org (kirk joena) Date: Mon, 07 Jul 2008 09:16:40 +0000 Subject: [ofa-general] to general Message-ID: <000501c8e021$047f4a61$bbb0d289@inmktgd> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avawewosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From ogerlitz at voltaire.com Mon Jul 7 04:36:02 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 07 Jul 2008 14:36:02 +0300 Subject: [ofa-general] Getting Started with RDMA In-Reply-To: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> Message-ID: <4871FFA2.1030700@voltaire.com> Adrien Guillon wrote: > I'm just getting started with RDMA... in particular my environment > consists of Voltaire Infiniband equipment... although I would like to > maintain platform independence as much as possible. I'm looking for a > user space library that supports atomic operations, and remote memory > transfers. I've looked a bit into librdmacm, basically looking at > rdma_cm.h for some general information. I haven't been able to find > much documentation on getting started with librdmacm, I suggest that you start with the man pages provided by the librdmacm-devel package Or. From sashak at voltaire.com Mon Jul 7 04:44:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 7 Jul 2008 14:44:08 +0300 Subject: [ofa-general] ibdmchk: how to consider multiple VLs while checking for credit loops ? In-Reply-To: <829ded920807070215g5e9ef6c4xf12082a69c09f93e@mail.gmail.com> References: <829ded920807070215g5e9ef6c4xf12082a69c09f93e@mail.gmail.com> Message-ID: <20080707114408.GD15449@sashak.voltaire.com> On 14:45 Mon 07 Jul , Keshetti Mahesh wrote: > > After the last message in the above thread (which is way back), I didn't see > anyone working on updating 'ibdmchk' to consider multiple VLs while checking > the network for credit loops. > > 'ibdmchk' requires "path<->VL" (not SL2VL) association information in order to > consider multIple VLs. But this information as of now is not dumped by OpenSM. > A proper format should be defined to dump this information by OpenSM as this > information is not straightforward like switch forwarding tables. > > Can anyone suggest me the best way to do it ? "SL for path" is requested using SA PathRecord requests. From command line you can do it with 'saquery PR'. Sasha From olaf.kirch at oracle.com Mon Jul 7 05:40:45 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Mon, 7 Jul 2008 14:40:45 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <200807071440.46502.olaf.kirch@oracle.com> On Thursday 03 July 2008 23:34:12 Jon Mason wrote: > This patch adds support for running RDS over iWARP adapters. It Hi Jon, I took your patch and tried to isolate the iWARP specific changes in bcopy mode, and roll them into a smaller patch that doesn't duplicate all the ib*.[hc] files. I also tried to come to some working solution for RDMA - as you can see from the deluge of messages I wrote on this :-) the approach you chose has some problems. Please take a look at the attached patch and let me know whether (a) bcopy mode works, and (b) if the rdma approach may work with iwarp nics. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax -------------- next part -------------- A non-text attachment was scrubbed... Name: rds-iwarp.patch Type: text/x-patch Size: 44157 bytes Desc: not available URL: From dorons at Voltaire.COM Mon Jul 7 07:08:49 2008 From: dorons at Voltaire.COM (Doron Shoham) Date: Mon, 07 Jul 2008 17:08:49 +0300 Subject: [ofa-general] Current list of Linux maintainers and their email info In-Reply-To: References: Message-ID: <48722371.5070900@Voltaire.COM> Hi Woody, Can you please add us under iSer: Doron Shoham, dorons at voltaire.com Eli Dorfman, elid at voltaire.com Thanks, Doron From ronli.voltaire at gmail.com Mon Jul 7 02:20:47 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Mon, 7 Jul 2008 12:20:47 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <200807061300.31811.jackm@dev.mellanox.co.il> References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> <200807061300.31811.jackm@dev.mellanox.co.il> Message-ID: <3b5e77ad0807070220i9f8659cm4f6e92e0286fcf5@mail.gmail.com> Hi, What do you think of the following approach? Instead of adding creation flags to the qp_init_attr, I can add a new verb: ibv_qp *create_qp_extended(struct ibv_pd *pd, struct ibv_qp_init_attr, *init_attr, enum ibv_qp_create_flags create_flags) I'm aware that adding a new verb isn't optimal, but at least we can avoid incrementing the libibverbs version. In your opinion, which way is better? Ron On Sun, Jul 6, 2008 at 1:00 PM, Jack Morgenstein wrote: > The ABI for ibv_create_qp (see file libibverbs/include/infiniband/kern-abi.h) > currently has 1 reserved byte. If we can use this for create flags, we don't need > to break the ABI, but clearly note that: > a. we will have space for only 8 flag bits -- not very many, they seem to be going > like hotcakes. > b. Kernel space create flags are 32 bits -- it may be weird to have userspace flags > be only 8 bits. We've also tried to keep bit offsets between kernel space and > userspace the same (e.g., see device capabilities flags) -- and this would > clearly make it impossible. > > On the other hand, I don't like the idea of adding a new verb for each new tweak. > > In the case of xrc receive qp's, new verbs were needed because the parameter list > for XRC receive qp's was very different (XRC domain handle and QP num, instead of > qp handle). > > In the case multicast loopback, a creation flag is most appropriate, because this > is really a qp-creation-only issue (the kernel core layer only needs this info at > QP creation time). > > Probably, the least short-term pain (no ABI change) would be to use the last reserved > create-qp ABI byte for flags, while defining the flags field in > ibv_qp_init_attr as uint32_t (preparing for more than 8 future flags). > > We would not have the same flag bits in user as in kernel, and if we at some point > needed more that 8 user-space qp create flags, we would increment the ABI at that point, > keep the originally defined 8 flags as they were, and just add a _u32 with additional flags. > > Adding the flags field to the qp_init_attr structure does, however, require incrementing the > libibverbs version to 1.2, necessitating an IBVERBS_1.1 compatibility layer. > > - Jack > > On Friday 04 July 2008 01:04, Or Gerlitz wrote: >> On 7/4/08, Roland Dreier wrote: >> >> > Sorry, I don't have a magic solution for you. To minimize pain about >> > ABI I think the only thing to do is minimize ABI changes. >> >> fair-enough, so lets just try to take ABI changes to minimum... >> >> > If a QP creation flag works for XRC then I would prefer not to add more >> > QP types I guess. One of the ways that the QP creation flags were >> > originally sold to me was that XRC needed them anyway. So I'm a bit >> > confused about where things have ended up. >> >> I can't take this, we think that creation flags can be a good way to >> go wrt minimizing ABI changes, I am not enough into the XRC impl to >> tell why creation flags are not used there. >> >> >> > I don't want to add both an XRC-specific mechanism and a more general >> > mechanism that XRC could have used but doesn't. So yes the "framework >> > Z" approach that works for both seems like the best way forward. >> >> OK, so do we agree that ABI wise one thing to handle is the new verb/s >> plus their associated data structures (in our case just create_flags >> field added to qp_init_attr) and second thing is placing the new verbs >> into the verbs context structure? >> >> Lets see what Jack says, we are fine with anything (yes) he would suggest. >> >> Or. >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From swise at opengridcomputing.com Mon Jul 7 07:36:32 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 09:36:32 -0500 Subject: [ofa-general] Re: [PATCH] krping: free child_cm_id in all error cases In-Reply-To: <20080706144031.GA20054@osc.edu> References: <20080706144031.GA20054@osc.edu> Message-ID: <487229F0.1000806@opengridcomputing.com> Applied. Thanks, Stevo. From swise at opengridcomputing.com Mon Jul 7 07:42:58 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 09:42:58 -0500 Subject: [ofa-general] Getting Started with RDMA In-Reply-To: <4871FFA2.1030700@voltaire.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> Message-ID: <48722B72.1050103@opengridcomputing.com> Or Gerlitz wrote: > Adrien Guillon wrote: >> I'm just getting started with RDMA... in particular my environment >> consists of Voltaire Infiniband equipment... although I would like to >> maintain platform independence as much as possible. I'm looking for >> a user space library that supports atomic operations, and remote >> memory transfers. I've looked a bit into librdmacm, basically >> looking at rdma_cm.h for some general information. I haven't been >> able to find much documentation on getting started with librdmacm, > I suggest that you start with the man pages provided by the > librdmacm-devel package > And the man pages provided by the libibverbs-devel package. Steve. From jon at opengridcomputing.com Mon Jul 7 08:12:23 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Mon, 7 Jul 2008 10:12:23 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> References: <20080703213412.GB32285@opengridcomputing.com> <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> Message-ID: <20080707151222.GA11451@opengridcomputing.com> On Fri, Jul 04, 2008 at 12:59:08AM +0300, Or Gerlitz wrote: > On 7/4/08, Jon Mason wrote: > > > The bulk of the code is a copy of the RDS IB with the changes necessary > > to get iWARP working. > > So the intension is to have two code bases "RDS/IB" and "RDS/iWARP"? > > if yes, why? the rds code was enhanced to support credit management > and my understanding is that this paves the way to have joint code, as > done for any other RDMA ULP that was ported to iWARP (rNFS, Open-MPI, > Lustre, etc) The plan was to create an iWARP specific module and once it has been verified to work well then push it into the IB module (and verify that it does not impact IB performance). I was erring on the side of caution and keeping it completely separate in this version and then merging as much common code as possible in a future version. > > The logic in the RDS connection setup to determine which method to use, > > requires a new function in the core IB logic to translate a given IP > > Address to the IB/iWARP device associated with it. The only place this > > can be determined is in the pre-existing structs in cma.c. > > .laddr_check is not used in this patchset, and I could not understand > its role from this description. Perhaps I misunderstand your comment, but I do believe laddr_check is being used in this patch for iWARP. See lines 323 and 354 of the patch. If you are asking what it is doing, it is using a new function I created in the infiniband core logic to determine what iWARP/IB device is tied to the given IP address and then checks to see if it is iWARP or IB. > > Olaf, if you would like to pull this into your personal tree, I can > > provide you with delta patches until it is accepted into the OFED 1.4 > > tree (assuming that you have no major problem with the code below). > > As its an RFC, lets have it go a little review before pulling it, > unless the intension is to put it in an experimental branch and let > people play with. Anyway, structured set of delta patches is always > prefered, for review, acceptance, debugging and maintainace reasons. Reviews are great! > > --- a/drivers/infiniband/core/cma.c > > +++ b/drivers/infiniband/core/cma.c > > > +struct ib_device *ipaddr_to_ibdev(u32 addr) > > is there a reason why rdma_resolve_addr(NULL, addr, timeout) wouldn't > do what ipaddr_to_ibdev does? There could very well be a different/better way of doing this. I could not find one at the time, but I will investigate your suggestion and see if I can get rid of this call. > Also, from quick looking I understand that this patchset adds some > sort of fmr pool for the new fmrs within the rds code, can this pool > be generalized and added to the core module such that other ULPs would > be able to enjoy it? The fast_reg_mr pool that is being created is very similar to the fmr pool that exists in the RDS IB module. I am not opposed to extrapolating this out, but are there any other users that would benefit from this? Thanks for the review :) Jon > > Or. From swise at opengridcomputing.com Mon Jul 7 08:47:13 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 10:47:13 -0500 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807062138.29103.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041207.40490.olaf.kirch@oracle.com> <200807041346.52282.olaf.kirch@oracle.com> <200807062138.29103.olaf.kirch@oracle.com> Message-ID: <48723A81.8020603@opengridcomputing.com> An HTML attachment was scrubbed... URL: From aj.guillon at gmail.com Mon Jul 7 08:29:14 2008 From: aj.guillon at gmail.com (Adrien Guillon) Date: Mon, 7 Jul 2008 11:29:14 -0400 Subject: ***SPAM*** Re: [ofa-general] Getting Started with RDMA In-Reply-To: <4871FFA2.1030700@voltaire.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> Message-ID: <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> I'll check out the man pages today. I was hoping for something that would give me a broad overview of the library implementation, the approach it takes and assumptions made. I'm a complete n00b with RDMA, but I would like to design a good C++ library to wrap the C implementation.... unless there is such a thing already. Is this forum appropriate for asking n00b questions as I get started programming with RDMA? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Mon Jul 7 09:11:24 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 09:11:24 -0700 Subject: ***SPAM*** Re: [ofa-general] Getting Started with RDMA In-Reply-To: <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> (Adrien Guillon's message of "Mon, 7 Jul 2008 11:29:14 -0400") References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> Message-ID: > I'll check out the man pages today. I was hoping for something that would > give me a broad overview of the library implementation, the approach it > takes and assumptions made. I'm a complete n00b with RDMA, but I would like > to design a good C++ library to wrap the C implementation.... unless there > is such a thing already. > Is this forum appropriate for asking n00b questions as I get started > programming with RDMA? Yes, it is the perfect place for such questions, but see below: | From: Bryan Green | Subject: [ofa-general] libibvpp - A libibverbs C++ wrapper library. | To: general at lists.openfabrics.org | Date: Tue, 13 May 2008 15:16:28 -0700 | | | I'd like to make an announcement about a recently released library. | I've released a C++ wrapper library for libibverbs, called libibvpp. | It is currently released under the NOSA (NASA Open Source Agreement) | license. For more information, please see the README (link below). | | I hope this library is of interest to the C++ programmers out there in the | OpenFabrics community. I'm also curious if there would be any interest in | hosting this project on the openfabrics website. | | Here is the library's current home: | | http://opensource.arc.nasa.gov/project/libivpp/ | | README: http://opensource.arc.nasa.gov/software/24/notes/ | Download: http://opensource.arc.nasa.gov/static/downloads/libibvpp-0.1.tar.gz From sashak at voltaire.com Mon Jul 7 09:35:34 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 7 Jul 2008 19:35:34 +0300 Subject: [ofa-general] [ANNOUNCE] IB management daily build tarballs Message-ID: <20080707163534.GL7229@sashak.voltaire.com> Hi, I'm starting to run daily builds for IB management packages (libibcommon, libibumad, libibmad, opensm, infiniband-diags). It will be placed at: http://www.openfabrics.org/downloads/management/daily First build tarballs are available already. Some details: 1. Files format are: ---.tar.gz Example: opensm-3.2.1-20080707-9d027ca.tar.gz 2. I think to run it via crontab (on my local machine, not on the OFA server). 3. New tarballs will be generated only for packages where actual changes have been made. Any feedback will be appreciated. Enjoy! Sasha From jon at opengridcomputing.com Mon Jul 7 09:55:55 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Mon, 7 Jul 2008 11:55:55 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807071440.46502.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807071440.46502.olaf.kirch@oracle.com> Message-ID: <20080707165555.GB11451@opengridcomputing.com> On Mon, Jul 07, 2008 at 02:40:45PM +0200, Olaf Kirch wrote: > On Thursday 03 July 2008 23:34:12 Jon Mason wrote: > > This patch adds support for running RDS over iWARP adapters. It > Hi Jon, > > I took your patch and tried to isolate the iWARP specific changes > in bcopy mode, and roll them into a smaller patch that doesn't duplicate > all the ib*.[hc] files. > > I also tried to come to some working solution for RDMA - as you can > see from the deluge of messages I wrote on this :-) the approach you > chose has some problems. > > Please take a look at the attached patch and let me know whether > (a) bcopy mode works, and (b) if the rdma approach may work with > iwarp nics. It doesn't seem to work. After looking through the code, the check_laddr will fail because it is still looking for the IB arp and not the inet arp needed by iWARP. I hacked around that but it is still not working. I'll look through the patch more and see if I can determine what is still breaking. Thanks, Jon > > Olaf > > -- > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax > From: Olaf Kirch > Subject: [PATCH RFC] RDS: Add iWARP Support > > This is based on the work posted by Jon Mason. It extracts > the iWARP-specific changes that are needed to support bcopy > mode (I hope I caught all of them). > > I also did some work on RDMA support. This is a lot harder, > because the interface and implementation were designed with > classic MRs in mind. However, I think the approach taken below > may result in a working approach (it's not working yet - I left > some blanks and BUG() asserts in there, because I wanted to get this > patch out as a RFC sooner rather than later). > > Also, this is a pretty large patch - it needs to be broken down into > half a dozen or so smaller functional changes for better review. > > Olaf > --- > net/rds/ib.c | 30 ++ > net/rds/ib.h | 55 ++++ > net/rds/ib_cm.c | 36 ++- > net/rds/ib_rdma.c | 610 +++++++++++++++++++++++++++++++++++++++++++++--------- > net/rds/ib_recv.c | 2 > net/rds/ib_send.c | 133 +++++++++++ > net/rds/message.c | 2 > net/rds/rdma.c | 17 - > net/rds/rdma.h | 7 > net/rds/rds.h | 4 > net/rds/send.c | 7 > 11 files changed, 778 insertions(+), 125 deletions(-) > > Index: build-2.6/net/rds/ib.c > =================================================================== > --- build-2.6.orig/net/rds/ib.c > +++ build-2.6/net/rds/ib.c > @@ -42,6 +42,7 @@ > > unsigned int fmr_pool_size = RDS_FMR_POOL_SIZE; > unsigned int fmr_message_size = RDS_FMR_SIZE + 1; /* +1 allows for unaligned MRs */ > +unsigned int fastreg_pool_size = RDS_FMR_POOL_SIZE; > > module_param(fmr_pool_size, int, 0444); > MODULE_PARM_DESC(fmr_pool_size, " Max number of fmr per HCA"); > @@ -85,21 +86,38 @@ void rds_ib_add_one(struct ib_device *de > rds_ibdev->fmr_page_size = 1 << rds_ibdev->fmr_page_shift; > rds_ibdev->fmr_page_mask = ~((u64) rds_ibdev->fmr_page_size - 1); > rds_ibdev->fmr_max_remaps = dev_attr->max_map_per_fmr?: 32; > - rds_ibdev->max_fmrs = dev_attr->max_fmr? > - min_t(unsigned int, dev_attr->max_fmr, fmr_pool_size) : > - fmr_pool_size; > + rds_ibdev->max_fmrs = dev_attr->max_fmr; > > rds_ibdev->dev = device; > rds_ibdev->pd = ib_alloc_pd(device); > if (IS_ERR(rds_ibdev->pd)) > goto free_dev; > > - rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, > - IB_ACCESS_LOCAL_WRITE); > + if (device->node_type != RDMA_NODE_RNIC) { > + rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, > + IB_ACCESS_LOCAL_WRITE); > + } else { > + /* Why does it have to have these permissions? */ > + rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, > + IB_ACCESS_REMOTE_READ | > + IB_ACCESS_REMOTE_WRITE | > + IB_ACCESS_LOCAL_WRITE); > + } > if (IS_ERR(rds_ibdev->mr)) > goto err_pd; > > - rds_ibdev->mr_pool = rds_ib_create_mr_pool(rds_ibdev); > + /* Create the MR pool. We choose different strategies for > + * MRs depending on the hardware. > + */ > + if (dev_attr->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS) { > + /* Use fast registrations */ > + rds_ibdev->mr_pool = rds_ib_create_fastreg_pool(rds_ibdev); > + rds_ibdev->use_fastreg = 1; > + } else { > + /* Default: use FMRs. Would be nice if there was > + * a capability flag to test for. */ > + rds_ibdev->mr_pool = rds_ib_create_mr_pool(rds_ibdev); > + } > if (IS_ERR(rds_ibdev->mr_pool)) { > rds_ibdev->mr_pool = NULL; > goto err_mr; > Index: build-2.6/net/rds/ib.h > =================================================================== > --- build-2.6.orig/net/rds/ib.h > +++ build-2.6/net/rds/ib.h > @@ -49,9 +49,51 @@ struct rds_ib_connect_private { > __be32 dp_credit; /* non-zero enables flow ctl */ > }; > > +struct rds_ib_scatterlist { > + struct scatterlist * list; > + unsigned int len; > + int dma_len; > +}; > + > +/* We need to post a LOCAL_INV request unless f_old_rkey > + * has this value. */ > +#define RDS_IB_INVALID_FASTREG_KEY 0 > + > +struct rds_ib_fastreg { > + atomic_t f_refcnt; > + unsigned int f_posted : 1, > + f_done : 1; > + > + u32 f_old_rkey; > + > + u32 f_rkey; > + unsigned int f_length; > + > + struct rds_ib_scatterlist f_sg; > + > + struct ib_fast_reg_page_list *f_page_list; > + unsigned int f_page_list_len; > + unsigned int f_page_shift; > + > +#if 0 > + u32 f_invalidate_rkey; > + struct ib_send_wr f_wr; > + wait_queue_head_t f_waitq; > + struct list_head f_list; > + unsigned int f_done; > + int f_status; > +#endif > + > + struct rds_ib_mr *f_mr; > +}; > + > struct rds_ib_send_work { > struct rds_message *s_rm; > + > + /* We should really put these into a union: */ > struct rds_rdma_op *s_op; > + struct rds_ib_fastreg *s_fastreg; > + > struct ib_send_wr s_wr; > struct ib_sge s_sge[RDS_IB_MAX_SGE]; > unsigned long s_queued; > @@ -86,6 +128,7 @@ struct rds_ib_connection { > struct rds_header *i_send_hdrs; > u64 i_send_hdrs_dma; > struct rds_ib_send_work *i_sends; > + struct list_head i_fastreg_pending; > > /* rx */ > struct mutex i_recv_mutex; > @@ -123,7 +166,9 @@ struct rds_ib_connection { > atomic_t i_credits; > > /* Protocol version specific information */ > - unsigned int i_flowctl : 1; /* enable/disable flow ctl */ > + unsigned int i_flowctl : 1, /* enable/disable flow ctl */ > + i_iwarp : 1, /* this is actually iWARP not IB */ > + i_fastreg : 1; /* use fastreg */ > > /* Batched completions */ > unsigned int i_unsignaled_wrs; > @@ -154,6 +199,7 @@ struct rds_ib_device { > unsigned int fmr_max_remaps; > unsigned int max_fmrs; > int max_sge; > + unsigned int use_fastreg : 1; > spinlock_t spinlock; > }; > > @@ -236,6 +282,7 @@ extern void rds_ib_remove_one(struct ib_ > extern struct ib_client rds_ib_client; > > extern unsigned int fmr_pool_size; > +extern unsigned int fastreg_pool_size; > extern unsigned int fmr_message_size; > > /* ib_cm.c */ > @@ -254,6 +301,7 @@ void __rds_ib_conn_error(struct rds_conn > /* ib_rdma.c */ > int ib_update_ipaddr_for_device(struct rds_ib_device *rds_ibdev, __be32 ipaddr); > struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *); > +struct rds_ib_mr_pool *rds_ib_create_fastreg_pool(struct rds_ib_device *); > void rds_ib_get_mr_info(struct rds_ib_device *rds_ibdev, struct rds_info_ib_connection *iinfo); > void rds_ib_destroy_mr_pool(struct rds_ib_mr_pool *); > void *rds_ib_get_mr(struct scatterlist *sg, unsigned long nents, > @@ -261,6 +309,10 @@ void *rds_ib_get_mr(struct scatterlist * > void rds_ib_sync_mr(void *trans_private, int dir); > void rds_ib_free_mr(void *trans_private, int invalidate); > void rds_ib_flush_mrs(void); > +struct rds_ib_fastreg *rds_ib_rdma_get_fastreg(struct rds_mr *); > +void rds_ib_fastreg_release(struct rds_ib_fastreg *frr); > +void rds_ib_local_inv_complete(struct rds_ib_fastreg *frr, int status); > +void rds_ib_fast_reg_complete(struct rds_ib_fastreg *frr, int status); > > /* ib_recv.c */ > int __init rds_ib_recv_init(void); > @@ -298,6 +350,7 @@ void rds_ib_send_cq_comp_handler(struct > void rds_ib_send_init_ring(struct rds_ib_connection *ic); > void rds_ib_send_clear_ring(struct rds_ib_connection *ic); > int rds_ib_xmit_rdma(struct rds_connection *conn, struct rds_rdma_op *op); > +int rds_ib_xmit_fastreg(struct rds_connection *conn, struct rds_mr *mr); > void rds_ib_send_add_credits(struct rds_connection *conn, unsigned int credits); > void rds_ib_advertise_credits(struct rds_connection *conn, unsigned int posted); > int rds_ib_send_grab_credits(struct rds_ib_connection *ic, u32 wanted, > Index: build-2.6/net/rds/ib_rdma.c > =================================================================== > --- build-2.6.orig/net/rds/ib_rdma.c > +++ build-2.6/net/rds/ib_rdma.c > @@ -45,20 +45,31 @@ extern struct list_head rds_ib_devices; > struct rds_ib_mr { > struct rds_ib_device *device; > struct rds_ib_mr_pool *pool; > - struct ib_fmr *fmr; > + > + spinlock_t lock; > + union { > + struct { > + struct ib_fmr *fmr; > + } ib; > + struct { > + struct ib_fast_reg_page_list *page_list; > + struct ib_mr *fastreg_mr; > + u32 rkey; > + struct rds_ib_fastreg *pending; > + } iwarp; > + } u; > struct list_head list; > unsigned int remap_count; > > - struct scatterlist * sg; > - unsigned int sg_len; > - u64 * dma; > - int sg_dma_len; > + struct rds_ib_scatterlist sg; > }; > > /* > * Our own little FMR pool > */ > struct rds_ib_mr_pool { > + struct rds_ib_device * device; > + > struct mutex flush_lock; /* serialize fmr invalidate */ > struct work_struct flush_worker; /* flush worker */ > > @@ -68,16 +79,57 @@ struct rds_ib_mr_pool { > struct list_head drop_list; /* MRs that have reached their max_maps limit */ > struct list_head free_list; /* unused MRs */ > struct list_head clean_list; /* unused & unamapped MRs */ > + struct list_head fastreg_list; /* pending fastreg's */ > atomic_t free_pinned; /* memory pinned by free MRs */ > + unsigned long max_message_size; /* in pages */ > unsigned long max_items; > unsigned long max_items_soft; > unsigned long max_free_pinned; > struct ib_fmr_attr fmr_attr; > + > + /* Dummy QP used to handle invalidate for fastreg */ > + struct ib_qp *qp; > + > + struct rds_ib_mr_pool_ops *op; > +}; > + > +struct rds_ib_mr_pool_ops { > + int (*init)(struct rds_ib_mr_pool *, struct rds_ib_mr *); > + int (*map)(struct rds_ib_mr_pool *pool, struct rds_ib_mr *ibmr, > + struct scatterlist *sg, unsigned int sg_len); > + void (*unmap)(struct rds_ib_mr_pool *, struct list_head *); > + void (*destroy)(struct rds_ib_mr_pool *, struct rds_ib_mr *); > }; > > static int rds_ib_flush_mr_pool(struct rds_ib_mr_pool *pool, int free_all); > static void rds_ib_teardown_mr(struct rds_ib_mr *ibmr); > static void rds_ib_mr_pool_flush_worker(struct work_struct *work); > +static int rds_ib_init_fmr(struct rds_ib_mr_pool *pool, struct rds_ib_mr *ibmr); > +static int rds_ib_map_fmr(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr, > + struct scatterlist *sg, unsigned int nents); > +static void rds_ib_unmap_fmr_list(struct rds_ib_mr_pool *pool, struct list_head *unmap_list); > +static void rds_ib_destroy_fmr(struct rds_ib_mr_pool *pool, struct rds_ib_mr *ibmr); > +static int rds_ib_init_fastreg(struct rds_ib_mr_pool *pool, struct rds_ib_mr *ibmr); > +static int rds_ib_map_fastreg(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr, > + struct scatterlist *sg, unsigned int nents); > +static void rds_ib_unmap_fastreg_list(struct rds_ib_mr_pool *pool, struct list_head *unmap_list); > +static void rds_ib_destroy_fastreg(struct rds_ib_mr_pool *pool, struct rds_ib_mr *ibmr); > + > +static struct rds_ib_mr_pool_ops rds_ib_fmr_pool_ops = { > + .init = rds_ib_init_fmr, > + .map = rds_ib_map_fmr, > + .unmap = rds_ib_unmap_fmr_list, > + .destroy = rds_ib_destroy_fmr, > +}; > + > +static struct rds_ib_mr_pool_ops rds_ib_fastreg_pool_ops = { > + .init = rds_ib_init_fastreg, > + .map = rds_ib_map_fastreg, > + .unmap = rds_ib_unmap_fastreg_list, > + .destroy = rds_ib_destroy_fastreg, > +}; > > int ib_update_ipaddr_for_device(struct rds_ib_device *rds_ibdev, __be32 ipaddr) > { > @@ -124,7 +176,158 @@ struct rds_ib_device* ib_get_device(__be > return NULL; > } > > -struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *rds_ibdev) > +static void rds_ib_set_scatterlist(struct rds_ib_scatterlist *sg, > + struct scatterlist *list, > + unsigned int sg_len, unsigned int sg_dma_len) > +{ > + sg->list = list; > + sg->len = sg_len; > + sg->dma_len = sg_dma_len; > +} > + > +static void rds_ib_rdma_drop_scatterlist(struct rds_ib_device *rds_ibdev, > + struct rds_ib_scatterlist *sg) > +{ > + if (sg->dma_len) { > + ib_dma_unmap_sg(rds_ibdev->dev, > + sg->list, sg->len, > + DMA_BIDIRECTIONAL); > + sg->dma_len = 0; > + } > + > + /* Release the s/g list */ > + if (sg->len) { > + unsigned int i; > + > + for (i = 0; i < sg->len; ++i) { > + struct page *page = sg_page(&sg->list[i]); > + > + /* FIXME we need a way to tell a r/w MR > + * from a r/o MR */ > + set_page_dirty(page); > + put_page(page); > + } > + kfree(sg->list); > + > + sg->list = NULL; > + sg->len = 0; > + } > +} > + > +/* > + * IB FMR handling > + */ > +static int rds_ib_init_fmr(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr) > +{ > + struct rds_ib_device *rds_ibdev = pool->device; > + struct ib_fmr *fmr; > + > + fmr = ib_alloc_fmr(rds_ibdev->pd, > + (IB_ACCESS_LOCAL_WRITE | > + IB_ACCESS_REMOTE_READ | > + IB_ACCESS_REMOTE_WRITE), > + &pool->fmr_attr); > + if (IS_ERR(fmr)) { > + int err = PTR_ERR(fmr); > + > + printk(KERN_WARNING "RDS/IB: ib_alloc_fmr failed (err=%d)\n", err); > + return err; > + } > + > + ibmr->u.ib.fmr = fmr; > + return 0; > +} > + > +static void rds_ib_unmap_fmr_list(struct rds_ib_mr_pool *pool, > + struct list_head *unmap_list) > +{ > + struct rds_ib_mr *ibmr; > + LIST_HEAD(fmr_list); > + int ret; > + > + /* String all ib_mr's onto one list and hand them to ib_unmap_fmr */ > + list_for_each_entry(ibmr, unmap_list, list) > + list_add(&ibmr->u.ib.fmr->list, &fmr_list); > + ret = ib_unmap_fmr(&fmr_list); > + if (ret) > + printk(KERN_WARNING "RDS/IB: ib_unmap_fmr failed (err=%d)\n", ret); > +} > + > +static void rds_ib_destroy_fmr(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr) > +{ > + if (ibmr->u.ib.fmr) > + ib_dealloc_fmr(ibmr->u.ib.fmr); > + ibmr->u.ib.fmr = NULL; > +} > + > +/* > + * iWARP fastreg handling > + */ > +static int rds_ib_init_fastreg(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr) > +{ > + struct rds_ib_device *rds_ibdev = pool->device; > + struct ib_mr *mr; > + > + mr = ib_alloc_fast_reg_mr(rds_ibdev->pd, pool->max_message_size); > + if (IS_ERR(mr)) { > + int err = PTR_ERR(mr); > + > + printk(KERN_WARNING "RDS/IWARP: ib_alloc_fast_reg_mr failed (err=%d)\n", err); > + return err; > + } > + > + ibmr->u.iwarp.rkey = RDS_IB_INVALID_FASTREG_KEY; > + ibmr->u.iwarp.fastreg_mr = mr; > + return 0; > +} > + > +static void rds_ib_unmap_fastreg_list(struct rds_ib_mr_pool *pool, > + struct list_head *unmap_list) > +{ > + LIST_HEAD(fmr_list); > + > + /* Batched invalidation of fastreg MRs. > + * Why do we do it this way, even though we could pipeline unmap > + * and remap? The reason is the application semantics - when the > + * application requests an invalidation of MRs, it expects all > + * previously released R_Keys to become invalid. > + * > + * If we implement MR reuse naively, we risk memory corruption > + * (this has actually been observed). So the default behavior > + * requires that a MR goes through an explicit unmap operation before > + * we can reuse it again. > + * > + * We could probably improve on this a little, by allowing immediate > + * reuse of a MR on the same socket (eg you could add small > + * cache of unused MRs to strct rds_socket - GET_MR could grab one > + * of these without requiring an explicit invalidate). > + */ > + > + /* Fill in the blanks: > + Go through the list of dirty MRs, and post LOCAL_INV WRs to the > + dummy pool->qp. When the completion for the last WR arrives, > + the CQ handler wakes up the caller. > + */ > + BUG(); /* not implemented yet. */ > +} > + > +static void rds_ib_destroy_fastreg(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr) > +{ > + if (ibmr->u.iwarp.page_list) > + ib_free_fast_reg_page_list(ibmr->u.iwarp.page_list); > + if (ibmr->u.iwarp.fastreg_mr) > + ib_dereg_mr(ibmr->u.iwarp.fastreg_mr); > + if (ibmr->u.iwarp.pending) > + rds_ib_fastreg_release(ibmr->u.iwarp.pending); > +} > + > +struct rds_ib_mr_pool *__rds_ib_create_mr_pool(struct rds_ib_device *rds_ibdev, > + unsigned int message_size, unsigned int pool_size, > + struct rds_ib_mr_pool_ops *ops) > { > struct rds_ib_mr_pool *pool; > > @@ -132,25 +335,68 @@ struct rds_ib_mr_pool *rds_ib_create_mr_ > if (!pool) > return ERR_PTR(-ENOMEM); > > + pool->device = rds_ibdev; > INIT_LIST_HEAD(&pool->free_list); > INIT_LIST_HEAD(&pool->drop_list); > INIT_LIST_HEAD(&pool->clean_list); > + INIT_LIST_HEAD(&pool->fastreg_list); > mutex_init(&pool->flush_lock); > spin_lock_init(&pool->list_lock); > INIT_WORK(&pool->flush_worker, rds_ib_mr_pool_flush_worker); > > - pool->fmr_attr.max_pages = fmr_message_size; > - pool->fmr_attr.max_maps = rds_ibdev->fmr_max_remaps; > - pool->fmr_attr.page_shift = rds_ibdev->fmr_page_shift; > - pool->max_free_pinned = rds_ibdev->max_fmrs * fmr_message_size / 4; > + pool->max_message_size = message_size; > + pool->max_items = pool_size; > + pool->max_free_pinned = pool->max_items * pool->max_message_size / 4; > > /* We never allow more than max_items MRs to be allocated. > * When we exceed more than max_items_soft, we start freeing > * items more aggressively. > * Make sure that max_items > max_items_soft > max_items / 2 > */ > - pool->max_items_soft = rds_ibdev->max_fmrs * 3 / 4; > - pool->max_items = rds_ibdev->max_fmrs; > + pool->max_items_soft = pool->max_items * 3 / 4; > + > + return pool; > +} > + > +struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *rds_ibdev) > +{ > + struct rds_ib_mr_pool *pool; > + unsigned int pool_size = fmr_pool_size; > + > + if (rds_ibdev->max_fmrs && rds_ibdev->max_fmrs < pool_size) > + pool_size = rds_ibdev->max_fmrs; > + > + pool = __rds_ib_create_mr_pool(rds_ibdev, fmr_message_size, pool_size, > + &rds_ib_fmr_pool_ops); > + > + if (!IS_ERR(pool)) { > + pool->fmr_attr.max_pages = pool->max_message_size; > + pool->fmr_attr.max_maps = rds_ibdev->fmr_max_remaps; > + pool->fmr_attr.page_shift = rds_ibdev->fmr_page_shift; > + } > + > + return pool; > +} > + > +struct rds_ib_mr_pool *rds_ib_create_fastreg_pool(struct rds_ib_device *rds_ibdev) > +{ > + struct rds_ib_mr_pool *pool; > + unsigned int pool_size = fmr_pool_size; > + > + if (rds_ibdev->max_fmrs && rds_ibdev->max_fmrs < pool_size) > + pool_size = rds_ibdev->max_fmrs; > + > + pool = __rds_ib_create_mr_pool(rds_ibdev, fmr_message_size, > + fastreg_pool_size, > + &rds_ib_fastreg_pool_ops); > + > + if (!IS_ERR(pool)) { > + /* Fill in the blanks: > + * create a dummy QP to which we can post LOCAL_INV > + * requests when invalidating MRs > + */ > + pool->qp = NULL; > + } > > return pool; > } > @@ -169,6 +415,10 @@ void rds_ib_destroy_mr_pool(struct rds_i > rds_ib_flush_mr_pool(pool, 1); > BUG_ON(atomic_read(&pool->item_count)); > BUG_ON(atomic_read(&pool->free_pinned)); > + > + if (pool->qp) > + ib_destroy_qp(pool->qp); > + > kfree(pool); > } > > @@ -227,77 +477,82 @@ static struct rds_ib_mr *rds_ib_alloc_fm > goto out_no_cigar; > } > > - ibmr->fmr = ib_alloc_fmr(rds_ibdev->pd, > - (IB_ACCESS_LOCAL_WRITE | > - IB_ACCESS_REMOTE_READ | > - IB_ACCESS_REMOTE_WRITE), > - &pool->fmr_attr); > - if (IS_ERR(ibmr->fmr)) { > - err = PTR_ERR(ibmr->fmr); > - ibmr->fmr = NULL; > - printk(KERN_WARNING "RDS/IB: ib_alloc_fmr failed (err=%d)\n", err); > + spin_lock_init(&ibmr->lock); > + > + err = pool->op->init(pool, ibmr); > + if (err) > goto out_no_cigar; > - } > > rds_ib_stats_inc(s_ib_rdma_mr_alloc); > return ibmr; > > out_no_cigar: > if (ibmr) { > - if (ibmr->fmr) > - ib_dealloc_fmr(ibmr->fmr); > + pool->op->destroy(pool, ibmr); > kfree(ibmr); > } > atomic_dec(&pool->item_count); > return ERR_PTR(err); > } > > -static int rds_ib_map_fmr(struct rds_ib_device *rds_ibdev, struct rds_ib_mr *ibmr, > - struct scatterlist *sg, unsigned int nents) > +static int rds_ib_count_dma_pages(struct rds_ib_device *rds_ibdev, > + struct scatterlist *sg, unsigned int sg_dma_len, > + unsigned int *lenp) > { > struct ib_device *dev = rds_ibdev->dev; > - struct scatterlist *scat = sg; > - u64 io_addr = 0; > - u64 *dma_pages; > - u32 len; > - int page_cnt, sg_dma_len; > - int i, j; > - int ret; > - > - sg_dma_len = ib_dma_map_sg(dev, sg, nents, > - DMA_BIDIRECTIONAL); > - if (unlikely(!sg_dma_len)) { > - printk(KERN_WARNING "RDS/IB: dma_map_sg failed!\n"); > - return -EBUSY; > - } > - > - len = 0; > - page_cnt = 0; > + unsigned int i, page_cnt = 0, len = 0; > > for (i = 0; i < sg_dma_len; ++i) { > - unsigned int dma_len = ib_sg_dma_len(dev, &scat[i]); > - u64 dma_addr = ib_sg_dma_address(dev, &scat[i]); > + unsigned int dma_len = ib_sg_dma_len(dev, &sg[i]); > + u64 dma_addr = ib_sg_dma_address(dev, &sg[i]); > > if (dma_addr & ~rds_ibdev->fmr_page_mask) { > if (i > 0) > return -EINVAL; > - else > - ++page_cnt; > + ++page_cnt; > } > if ((dma_addr + dma_len) & ~rds_ibdev->fmr_page_mask) { > if (i < sg_dma_len - 1) > return -EINVAL; > - else > - ++page_cnt; > + ++page_cnt; > } > > len += dma_len; > } > > page_cnt += len >> rds_ibdev->fmr_page_shift; > - if (page_cnt > fmr_message_size) > + if (page_cnt > rds_ibdev->mr_pool->max_message_size) > return -EINVAL; > > + return page_cnt; > +} > + > +static int rds_ib_map_fmr(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr, > + struct scatterlist *sg, unsigned int nents) > +{ > + struct rds_ib_device *rds_ibdev = pool->device; > + struct ib_device *dev = rds_ibdev->dev; > + struct scatterlist *scat = sg; > + u64 io_addr = 0; > + u64 *dma_pages; > + int page_cnt, sg_dma_len; > + int i, j; > + int ret; > + > + sg_dma_len = ib_dma_map_sg(dev, sg, nents, > + DMA_BIDIRECTIONAL); > + if (unlikely(!sg_dma_len)) { > + printk(KERN_WARNING "RDS/IB: dma_map_sg failed!\n"); > + return -EBUSY; > + } > + > + /* FIXME: when returning an error, we need to unmap the SG */ > + > + page_cnt = rds_ib_count_dma_pages(rds_ibdev, sg, sg_dma_len, NULL); > + if (page_cnt < 0) > + return page_cnt; > + > dma_pages = kmalloc(sizeof(u64) * page_cnt, GFP_ATOMIC); > if (!dma_pages) > return -ENOMEM; > @@ -312,7 +567,7 @@ static int rds_ib_map_fmr(struct rds_ib_ > (dma_addr & rds_ibdev->fmr_page_mask) + j; > } > > - ret = ib_map_phys_fmr(ibmr->fmr, > + ret = ib_map_phys_fmr(ibmr->u.ib.fmr, > dma_pages, page_cnt, io_addr); > if (ret) > goto out; > @@ -321,9 +576,9 @@ static int rds_ib_map_fmr(struct rds_ib_ > * safely tear down the old mapping. */ > rds_ib_teardown_mr(ibmr); > > - ibmr->sg = scat; > - ibmr->sg_len = nents; > - ibmr->sg_dma_len = sg_dma_len; > + ibmr->sg.list = scat; > + ibmr->sg.len = nents; > + ibmr->sg.dma_len = sg_dma_len; > ibmr->remap_count++; > > rds_ib_stats_inc(s_ib_rdma_mr_used); > @@ -335,6 +590,192 @@ out: > return ret; > } > > +static int rds_ib_map_fastreg(struct rds_ib_mr_pool *pool, > + struct rds_ib_mr *ibmr, > + struct scatterlist *sg, unsigned int sg_len) > +{ > + struct rds_ib_device *rds_ibdev = pool->device; > + struct ib_device *dev = rds_ibdev->dev; > + struct ib_fast_reg_page_list *page_list = NULL; > + struct rds_ib_fastreg *frr; > + unsigned int len; > + int i, j, page_cnt, sg_dma_len = 0; > + int ret; > + > + BUG_ON(ibmr->u.iwarp.pending); > + > + page_list = ib_alloc_fast_reg_page_list(rds_ibdev->dev, pool->max_message_size); > + if (IS_ERR(page_list)) { > + ret = PTR_ERR(page_list); > + page_list = NULL; > + > + printk(KERN_WARNING "RDS/iWARP: ib_alloc_fast_reg_page_list failed (err=%d)\n", ret); > + return ret; > + } > + > + sg_dma_len = ib_dma_map_sg(dev, sg, sg_len, DMA_BIDIRECTIONAL); > + if (unlikely(!sg_dma_len)) { > + printk(KERN_WARNING "RDS/iWARP: dma_map_sg failed!\n"); > + ret = -EBUSY; > + goto out; > + } > + > + page_cnt = rds_ib_count_dma_pages(rds_ibdev, sg, sg_dma_len, &len); > + if (page_cnt < 0) { > + ret = page_cnt; > + goto out; > + } > + > + page_cnt = 0; > + for (i = 0; i < sg_dma_len; ++i) { > + unsigned int dma_len = ib_sg_dma_len(dev, &sg[i]); > + u64 dma_addr = ib_sg_dma_address(dev, &sg[i]); > + > + for (j = 0; j < dma_len; j += rds_ibdev->fmr_page_size) > + page_list->page_list[page_cnt++] = > + (dma_addr & rds_ibdev->fmr_page_mask) + j; > + } > + > + /* Allocate the fastreg request structure */ > + frr = kzalloc(sizeof(*frr), GFP_KERNEL); > + if (!frr) { > + ret = -ENOMEM; > + goto out; > + } > + > + ib_update_fast_reg_key(ibmr->u.iwarp.fastreg_mr, ibmr->remap_count++); > + > + /* Build the fastreg WR */ > + frr->f_mr = ibmr; > + rds_ib_set_scatterlist(&frr->f_sg, sg, sg_len, sg_dma_len); > + frr->f_length = len; > + frr->f_rkey = ibmr->u.iwarp.fastreg_mr->rkey; > + frr->f_page_list = page_list; > + frr->f_page_list_len = sg_dma_len; > + frr->f_page_shift = rds_ibdev->fmr_page_shift; > + > + frr->f_old_rkey = ibmr->u.iwarp.rkey; > + > + /* Attach the fastreg info to the MR */ > + atomic_set(&frr->f_refcnt, 1); > + ibmr->u.iwarp.pending = frr; > + > + rds_ib_stats_inc(s_ib_rdma_mr_used); > + ret = 0; > + > +out: > + if (ret) { > + ib_free_fast_reg_page_list(page_list); > + if (sg_dma_len) > + ib_dma_unmap_sg(dev, sg, sg_dma_len, DMA_BIDIRECTIONAL); > + } > + > + return ret; > +} > + > +struct rds_ib_fastreg *rds_ib_rdma_get_fastreg(struct rds_mr *mr) > +{ > + struct rds_ib_mr *ibmr = mr->r_trans_private; > + struct rds_ib_fastreg *frr; > + unsigned long flags; > + > + spin_lock_irqsave(&ibmr->lock, flags); > + frr = ibmr->u.iwarp.pending; > + if (frr) { > + /* FIXME: we need to mark the frr as "locked" > + * to prevent FREE_MR from trashing the MR > + * as long as the fastreg is on the queue */ > + atomic_inc(&frr->f_refcnt); > + } > + spin_unlock_irqrestore(&ibmr->lock, flags); > + > + return frr; > +} > + > +void rds_ib_fastreg_release(struct rds_ib_fastreg *frr) > +{ > + struct rds_ib_device *rds_ibdev = NULL; > + > + if (atomic_dec_and_test(&frr->f_refcnt)) { > + ib_free_fast_reg_page_list(frr->f_page_list); > + BUG(); /* FIXME: obtain rds_ibdev */ > + rds_ib_rdma_drop_scatterlist(rds_ibdev, &frr->f_sg); > + kfree(frr); > + } > +} > + > +/* > + * These functions are called back from the send CQ handler > + * when the LOCAL_INV or FAST_REG_MR WRs complete. > + */ > +void rds_ib_local_inv_complete(struct rds_ib_fastreg *frr, int status) > +{ > + struct rds_ib_mr *ibmr = frr->f_mr; > + > + spin_lock(&ibmr->lock); > + if (ibmr->u.iwarp.pending != frr) > + goto out_unlock; > + > + if (status != IB_WC_SUCCESS) { > + /* Yikes. Invalidation failed. What can we do but complain? */ > + printk(KERN_NOTICE "RDS/iWARP: Unable to invalidate fastreg MR.\n"); > + goto out_unlock; > + } > + > + if (frr->f_old_rkey == ibmr->u.iwarp.rkey) { > + ibmr->u.iwarp.rkey = 0; > + /* Now we can unpin any memory pinned for this MR. */ > + rds_ib_teardown_mr(ibmr); > + } > + frr->f_old_rkey = RDS_IB_INVALID_FASTREG_KEY; > + > +out_unlock: > + spin_unlock(&ibmr->lock); > + > + /* The WR owned a reference to this frr. Drop it */ > + rds_ib_fastreg_release(frr); > +} > + > +void rds_ib_fast_reg_complete(struct rds_ib_fastreg *frr, int status) > +{ > + struct rds_ib_mr *ibmr = frr->f_mr; > + > + spin_lock(&ibmr->lock); > + > + /* Technically, this would be a bug */ > + if (ibmr->u.iwarp.pending != frr) > + goto out_unlock; > + > + if (status != IB_WC_SUCCESS) { > + /* Yikes. We were unable to register the application's > + * memory. We have no way of notifying the application. > + * We could probably tear down the QP and cry uncle, but > + * the SEND may already have gone out. > + * The only solace is that the RDMA initiated by the remote > + * will fail, because the key isn't valid. > + */ > + if (printk_ratelimit()) > + printk(KERN_NOTICE "RDS/iWARP: Unable to " > + "perform fast memory registration.\n"); > + goto out_unlock; > + } > + > + ibmr->sg = frr->f_sg; > + ibmr->u.iwarp.page_list = frr->f_page_list; > + ibmr->u.iwarp.rkey = frr->f_rkey; > + > + /* Detach frr from MR. We still have at least one ref after this */ > + ibmr->u.iwarp.pending = NULL; > + rds_ib_fastreg_release(frr); > + frr->f_done = 1; > + > +out_unlock: > + spin_unlock(&ibmr->lock); > + > + /* The WR owned a reference to this frr. Drop it */ > + rds_ib_fastreg_release(frr); > +} > + > void rds_ib_sync_mr(void *trans_private, int direction) > { > struct rds_ib_mr *ibmr = trans_private; > @@ -342,49 +783,24 @@ void rds_ib_sync_mr(void *trans_private, > > switch (direction) { > case DMA_FROM_DEVICE: > - ib_dma_sync_sg_for_cpu(rds_ibdev->dev, ibmr->sg, > - ibmr->sg_dma_len, DMA_BIDIRECTIONAL); > + ib_dma_sync_sg_for_cpu(rds_ibdev->dev, ibmr->sg.list, > + ibmr->sg.dma_len, DMA_BIDIRECTIONAL); > break; > case DMA_TO_DEVICE: > - ib_dma_sync_sg_for_device(rds_ibdev->dev, ibmr->sg, > - ibmr->sg_dma_len, DMA_BIDIRECTIONAL); > + ib_dma_sync_sg_for_device(rds_ibdev->dev, ibmr->sg.list, > + ibmr->sg.dma_len, DMA_BIDIRECTIONAL); > break; > } > } > > static void __rds_ib_teardown_mr(struct rds_ib_mr *ibmr) > { > - struct rds_ib_device *rds_ibdev = ibmr->device; > - > - if (ibmr->sg_dma_len) { > - ib_dma_unmap_sg(rds_ibdev->dev, > - ibmr->sg, ibmr->sg_len, > - DMA_BIDIRECTIONAL); > - ibmr->sg_dma_len = 0; > - } > - > - /* Release the s/g list */ > - if (ibmr->sg_len) { > - unsigned int i; > - > - for (i = 0; i < ibmr->sg_len; ++i) { > - struct page *page = sg_page(&ibmr->sg[i]); > - > - /* FIXME we need a way to tell a r/w MR > - * from a r/o MR */ > - set_page_dirty(page); > - put_page(page); > - } > - kfree(ibmr->sg); > - > - ibmr->sg = NULL; > - ibmr->sg_len = 0; > - } > + rds_ib_rdma_drop_scatterlist(ibmr->device, &ibmr->sg); > } > > void rds_ib_teardown_mr(struct rds_ib_mr *ibmr) > { > - unsigned int pinned = ibmr->sg_len; > + unsigned int pinned = ibmr->sg.len; > > __rds_ib_teardown_mr(ibmr); > if (pinned) { > @@ -419,7 +835,6 @@ int rds_ib_flush_mr_pool(struct rds_ib_m > { > struct rds_ib_mr *ibmr, *next; > LIST_HEAD(unmap_list); > - LIST_HEAD(fmr_list); > unsigned long unpinned = 0; > unsigned long flags; > unsigned int nfreed = 0, ncleaned = 0, free_goal; > @@ -443,21 +858,17 @@ int rds_ib_flush_mr_pool(struct rds_ib_m > if (list_empty(&unmap_list)) > goto out; > > - /* String all ib_mr's onto one list and hand them to ib_unmap_fmr */ > - list_for_each_entry(ibmr, &unmap_list, list) > - list_add(&ibmr->fmr->list, &fmr_list); > - ret = ib_unmap_fmr(&fmr_list); > - if (ret) > - printk(KERN_WARNING "RDS/IB: ib_unmap_fmr failed (err=%d)\n", ret); > + /* Batched invalidate of dirty MRs: */ > + pool->op->unmap(pool, &unmap_list); > > /* Now we can destroy the DMA mapping and unpin any pages */ > list_for_each_entry_safe(ibmr, next, &unmap_list, list) { > - unpinned += ibmr->sg_len; > + unpinned += ibmr->sg.len; > __rds_ib_teardown_mr(ibmr); > if (nfreed < free_goal || ibmr->remap_count >= pool->fmr_attr.max_maps) { > rds_ib_stats_inc(s_ib_rdma_mr_free); > list_del(&ibmr->list); > - ib_dealloc_fmr(ibmr->fmr); > + pool->op->destroy(pool, ibmr); > kfree(ibmr); > nfreed++; > } > @@ -491,7 +902,7 @@ void rds_ib_free_mr(void *trans_private, > struct rds_ib_mr_pool *pool = rds_ibdev->mr_pool; > unsigned long flags; > > - rdsdebug("RDS/IB: free_mr nents %u\n", ibmr->sg_len); > + rdsdebug("RDS/IB: free_mr nents %u\n", ibmr->sg.len); > > /* Return it to the pool's free list */ > spin_lock_irqsave(&pool->list_lock, flags); > @@ -500,7 +911,7 @@ void rds_ib_free_mr(void *trans_private, > } else { > list_add(&ibmr->list, &pool->free_list); > } > - atomic_add(ibmr->sg_len, &pool->free_pinned); > + atomic_add(ibmr->sg.len, &pool->free_pinned); > atomic_inc(&pool->dirty_count); > spin_unlock_irqrestore(&pool->list_lock, flags); > > @@ -536,6 +947,7 @@ void *rds_ib_get_mr(struct scatterlist * > __be32 ip_addr, u32 *key_ret) > { > struct rds_ib_device *rds_ibdev; > + struct rds_ib_mr_pool *pool; > struct rds_ib_mr *ibmr = NULL; > int ret; > > @@ -545,7 +957,7 @@ void *rds_ib_get_mr(struct scatterlist * > goto out; > } > > - if (!rds_ibdev->mr_pool) { > + if (!(pool = rds_ibdev->mr_pool)) { > ret = -ENODEV; > goto out; > } > @@ -554,9 +966,9 @@ void *rds_ib_get_mr(struct scatterlist * > if (IS_ERR(ibmr)) > return ibmr; > > - ret = rds_ib_map_fmr(rds_ibdev, ibmr, sg, nents); > + ret = pool->op->map(pool, ibmr, sg, nents); > if (ret == 0) > - *key_ret = ibmr->fmr->rkey; > + *key_ret = ibmr->u.ib.fmr->rkey; > else > printk(KERN_WARNING "RDS/IB: map_fmr failed (errno=%d)\n", ret); > > Index: build-2.6/net/rds/rdma.c > =================================================================== > --- build-2.6.orig/net/rds/rdma.c > +++ build-2.6/net/rds/rdma.c > @@ -116,11 +116,8 @@ static void rds_destroy_mr(struct rds_mr > mr->r_trans->free_mr(trans_private, mr->r_invalidate); > } > > -static void rds_mr_put(struct rds_mr *mr) > +void __rds_put_mr_final(struct rds_mr *mr) > { > - if (!atomic_dec_and_test(&mr->r_refcount)) > - return; > - > rds_destroy_mr(mr); > kfree(mr); > } > @@ -169,7 +166,7 @@ static int rds_pin_pages(unsigned long u > } > > static int __rds_rdma_map(struct rds_sock *rs, struct rds_get_mr_args *args, > - u64 *cookie_ret) > + u64 *cookie_ret, struct rds_mr **mr_ret) > { > struct rds_mr *mr = NULL, *found; > unsigned int nr_pages; > @@ -297,6 +294,10 @@ static int __rds_rdma_map(struct rds_soc > > rdsdebug("RDS: get_mr key is %x\n", mr->r_key); > > + if (mr_ret) { > + atomic_inc(&mr->r_refcount); > + *mr_ret = mr; > + } > ret = 0; > out: > if (pages) > @@ -317,7 +318,7 @@ int rds_get_mr(struct rds_sock *rs, char > sizeof(struct rds_get_mr_args))) > return -EFAULT; > > - return __rds_rdma_map(rs, &args, NULL); > + return __rds_rdma_map(rs, &args, NULL, NULL); > } > > /* > @@ -655,7 +656,7 @@ int rds_cmsg_rdma_dest(struct rds_sock * > > if (mr) { > mr->r_trans->sync_mr(mr->r_trans_private, DMA_TO_DEVICE); > - rds_mr_put(mr); > + rm->m_rdma_mr = mr; > } > return err; > } > @@ -673,5 +674,5 @@ int rds_cmsg_rdma_map(struct rds_sock *r > || rm->m_rdma_cookie != 0) > return -EINVAL; > > - return __rds_rdma_map(rs, CMSG_DATA(cmsg), &rm->m_rdma_cookie); > + return __rds_rdma_map(rs, CMSG_DATA(cmsg), &rm->m_rdma_cookie, &rm->m_rdma_mr); > } > Index: build-2.6/net/rds/rds.h > =================================================================== > --- build-2.6.orig/net/rds/rds.h > +++ build-2.6/net/rds/rds.h > @@ -30,6 +30,7 @@ > */ > #define RDS_IB_PORT 18635 > #define RDS_TCP_PORT 18636 > +#define RDS_IWARP_PORT 18637 > > #ifndef AF_RDS > #define AF_RDS 28 /* Reliable Datagram Socket */ > @@ -60,6 +61,7 @@ > /* XXX crap, we need to worry about this conflicting too */ > #define SYSCTL_NET_RDS 9912 > #define SYSCTL_NET_RDS_IB 100 > +#define SYSCTL_NET_RDS_IWARP 101 > > #ifdef DEBUG > #define rdsdebug(fmt, args...) pr_debug("%s(): " fmt, __func__ , ##args) > @@ -282,6 +284,7 @@ struct rds_incoming { > #define RDS_MSG_RETRANSMITTED 5 > #define RDS_MSG_MAPPED 6 > #define RDS_MSG_PAGEVEC 7 > +#define RDS_MSG_FASTREG_POSTED 8 > > struct rds_message { > atomic_t m_refcount; > @@ -301,6 +304,7 @@ struct rds_message { > struct rds_sock *m_rs; > struct rds_rdma_op *m_rdma_op; > rds_rdma_cookie_t m_rdma_cookie; > + struct rds_mr *m_rdma_mr; > unsigned int m_nents; > unsigned int m_count; > struct scatterlist m_sg[0]; > Index: build-2.6/net/rds/ib_cm.c > =================================================================== > --- build-2.6.orig/net/rds/ib_cm.c > +++ build-2.6/net/rds/ib_cm.c > @@ -142,16 +142,19 @@ static void rds_ib_cm_fill_conn_param(st > struct rds_ib_connect_private *dp, > u32 protocol_version) > { > + struct rds_ib_connection *ic = conn->c_transport_data; > + > memset(conn_param, 0, sizeof(struct rdma_conn_param)); > /* XXX tune these? */ > conn_param->responder_resources = 1; > conn_param->initiator_depth = 1; > - conn_param->retry_count = 7; > - conn_param->rnr_retry_count = 7; > > - if (dp) { > - struct rds_ib_connection *ic = conn->c_transport_data; > + if (!ic->i_iwarp) { > + conn_param->retry_count = 7; > + conn_param->rnr_retry_count = 7; > + } > > + if (dp) { > memset(dp, 0, sizeof(*dp)); > dp->dp_saddr = conn->c_laddr; > dp->dp_daddr = conn->c_faddr; > @@ -288,7 +291,7 @@ static int rds_ib_setup_qp(struct rds_co > */ > ret = rdma_create_qp(ic->i_cm_id, ic->i_pd, &attr); > if (ret) { > - rdsdebug("ib_req_notify_cq failed: %d\n", ret); > + rdsdebug("rdma_create_qp failed: %d\n", ret); > goto out; > } > > @@ -442,6 +445,12 @@ static int rds_ib_cm_handle_connect(stru > ic->i_cm_id = cm_id; > cm_id->context = conn; > > + rds_ibdev = ib_get_client_data(cm_id->device, &rds_ib_client); > + > + /* Remember whether this is IB or iWARP */ > + ic->i_iwarp = (cm_id->device->node_type == RDMA_NODE_RNIC); > + ic->i_fastreg = rds_ibdev->use_fastreg; > + > /* We got halfway through setting up the ib_connection, if we > * fail now, we have to take the long route out of this mess. */ > destroy = 0; > @@ -462,7 +471,6 @@ static int rds_ib_cm_handle_connect(stru > } > > /* update ib_device with this local ipaddr */ > - rds_ibdev = ib_get_client_data(ic->i_cm_id->device, &rds_ib_client); > ib_update_ipaddr_for_device(rds_ibdev, dp->dp_saddr); > > return 0; > @@ -616,6 +624,17 @@ int rds_ib_conn_connect(struct rds_conne > src.sin_addr.s_addr = (__force u32)conn->c_laddr; > src.sin_port = (__force u16)htons(0); > > + /* First, bind to the local address and device. */ > + ret = rdma_bind_addr(ic->i_cm_id, (struct sockaddr *) &src); > + if (ret) { > + rdsdebug("rdma_bind_addr(%u.%u.%u.%u) failed: %d\n", > + NIPQUAD(conn->c_laddr), ret); > + goto out; > + } > + > + /* Now check the device type and set i_iwarp */ > + ic->i_iwarp = (ic->i_cm_id->device->node_type == RDMA_NODE_RNIC); > + > dest.sin_family = AF_INET; > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > dest.sin_port = (__force u16)htons(RDS_IB_PORT); > @@ -662,8 +681,9 @@ void rds_ib_conn_shutdown(struct rds_con > " cm: %p err %d\n", ic->i_cm_id, err); > } > > - /* Always move the QP to error state */ > - if (ic->i_cm_id->qp) { > + /* For IB, we have to move the QP to error state. > + * This is not needed for iWARP */ > + if (ic->i_cm_id->qp && !ic->i_iwarp) { > qp_attr.qp_state = IB_QPS_ERR; > err = ib_modify_qp(ic->i_cm_id->qp, &qp_attr, IB_QP_STATE); > if (err) { > Index: build-2.6/net/rds/ib_send.c > =================================================================== > --- build-2.6.orig/net/rds/ib_send.c > +++ build-2.6/net/rds/ib_send.c > @@ -165,6 +165,8 @@ void rds_ib_send_clear_ring(struct rds_i > rds_ib_send_unmap_rm(ic, send, IB_WC_WR_FLUSH_ERR); > if (send->s_op) > rds_ib_send_unmap_rdma(ic, send->s_op); > + if (send->s_fastreg) > + rds_ib_fastreg_release(send->s_fastreg); > } > } > > @@ -195,7 +197,7 @@ void rds_ib_send_cq_comp_handler(struct > while (ib_poll_cq(cq, 1, &wc) > 0 ) { > rdsdebug("wc wr_id 0x%llx status %u byte_len %u imm_data %u\n", > (unsigned long long)wc.wr_id, wc.status, wc.byte_len, > - be32_to_cpu(wc.imm_data)); > + be32_to_cpu(wc.ex.imm_data)); > rds_ib_stats_inc(s_ib_tx_cq_event); > > if (wc.wr_id == RDS_IB_ACK_WR_ID) { > @@ -223,6 +225,16 @@ void rds_ib_send_cq_comp_handler(struct > /* Nothing to be done - the SG list will be unmapped > * when the SEND completes. */ > break; > + case IB_WR_LOCAL_INV: > + /* We invalidated an r_key. the caller may want to > + * learn about this. */ > + if (send->s_fastreg) > + rds_ib_local_inv_complete(send->s_fastreg, wc.status); > + break; > + case IB_WR_FAST_REG_MR: > + if (send->s_fastreg) > + rds_ib_fast_reg_complete(send->s_fastreg, wc.status); > + break; > default: > if (printk_ratelimit()) > printk(KERN_NOTICE > @@ -261,7 +273,7 @@ void rds_ib_send_cq_comp_handler(struct > * queue_delay_work will not do anything if the work > * struct is already queued, so we need to cancel it first. > */ > - cancel_delayed_work(&conn->c_send_w); > + cancel_delayed_work(&conn->c_send_w); /* FIXME barf */ > queue_delayed_work(rds_wq, &conn->c_send_w, 0); > } > > @@ -490,6 +502,21 @@ int rds_ib_xmit(struct rds_connection *c > else > i = ceil(be32_to_cpu(rm->m_inc.i_hdr.h_len), RDS_FRAG_SIZE); > > + /* Fastreg support */ > + if (rds_rdma_cookie_key(rm->m_rdma_cookie) > + && ic->i_fastreg > + && !test_bit(RDS_MSG_FASTREG_POSTED, &rm->m_flags)) { > + ret = rds_ib_xmit_fastreg(conn, rm->m_rdma_mr); > + if (ret) > + goto out; > + > + /* We don't release the fastreg yet - we can only > + * do that when it has completed. If the connection > + * goes down, and we re-queue the message, we would > + * have to retry the registration. */ > + set_bit(RDS_MSG_FASTREG_POSTED, &rm->m_flags); > + } > + > work_alloc = rds_ib_ring_alloc(&ic->i_send_ring, i, &pos); > if (work_alloc == 0) { > set_bit(RDS_LL_SEND_FULL, &conn->c_flags); > @@ -849,6 +876,108 @@ out: > return ret; > } > > +static int __rds_ib_xmit_fastreg(struct rds_connection *conn, > + struct rds_ib_fastreg *frr) > +{ > + struct rds_ib_connection *ic = conn->c_transport_data; > + struct rds_ib_send_work *send = NULL; > + struct rds_ib_send_work *first; > + struct ib_send_wr *failed_wr; > + u32 pos; > + u32 work_alloc = 0; > + int ret; > + int num_wrs; > + > + /* > + * Perform 2 WRs for the fast_reg_mr's and chain them together. The > + * first WR is used to invalidate the old rkey, and the second WR is > + * used to define the new fast_reg_mr request. Each individual page > + * in the sg list is added to the fast reg page list and placed > + * inside the fast_reg_mr WR. The key used is a rolling 8bit > + * counter, which should guarantee uniqueness. > + */ > + num_wrs = 0; > + if (frr->f_old_rkey != RDS_IB_INVALID_FASTREG_KEY) > + num_wrs++; > + if (frr->f_page_list) > + num_wrs++; > + if (!num_wrs) > + return 0; > + > + work_alloc = rds_ib_ring_alloc(&ic->i_send_ring, num_wrs, &pos); > + if (work_alloc != num_wrs) { > + rds_ib_ring_unalloc(&ic->i_send_ring, work_alloc); > + rds_ib_stats_inc(s_ib_tx_ring_full); > + ret = -ENOMEM; > + goto out; > + } > + > + first = send = &ic->i_sends[pos]; > + > + if (frr->f_old_rkey != RDS_IB_INVALID_FASTREG_KEY) { > + memset(send, 0, sizeof(*send)); > + send->s_wr.opcode = IB_WR_LOCAL_INV; > + send->s_wr.ex.invalidate_rkey = frr->f_old_rkey; > + send->s_fastreg = frr; > + send->s_queued = jiffies; > + > + /* Get the next WR */ > + pos = (pos + 1) % ic->i_send_ring.w_nr; > + send = &ic->i_sends[pos]; > + } > + > + if (frr->f_page_list) { > + memset(send, 0, sizeof(*send)); > + send->s_wr.opcode = IB_WR_FAST_REG_MR; > + send->s_wr.wr.fast_reg.length = frr->f_length; > + send->s_wr.wr.fast_reg.rkey = frr->f_rkey; > + send->s_wr.wr.fast_reg.page_list = frr->f_page_list; > + send->s_wr.wr.fast_reg.page_list_len = frr->f_page_list_len; > + send->s_wr.wr.fast_reg.page_shift = frr->f_page_shift; > + send->s_wr.wr.fast_reg.access_flags = IB_ACCESS_LOCAL_WRITE | > + IB_ACCESS_REMOTE_READ | > + IB_ACCESS_REMOTE_WRITE; > + send->s_fastreg = frr; > + send->s_queued = jiffies; > + } > + > + atomic_add(num_wrs, &frr->f_refcnt); > + > + /* Chain the two WRs together */ > + if (num_wrs == 2) > + first->s_wr.next = &send->s_wr; > + > + failed_wr = &first->s_wr; > + ret = ib_post_send(ic->i_cm_id->qp, &first->s_wr, &failed_wr); > + > + rdsdebug("ic %p first %p (wr %p) ret %d wr %p\n", ic, > + first, &first->s_wr, ret, failed_wr); > + BUG_ON(failed_wr != &first->s_wr); > + if (ret) { > + printk(KERN_WARNING "RDS/IB: fastreg ib_post_send to %u.%u.%u.%u " > + "returned %d\n", NIPQUAD(conn->c_faddr), ret); > + while (num_wrs--) > + rds_ib_fastreg_release(frr); > + rds_ib_ring_unalloc(&ic->i_send_ring, 2); > + return ret; > + } > + > +out: > + return ret; > +} > + > +int rds_ib_xmit_fastreg(struct rds_connection *conn, struct rds_mr *mr) > +{ > + struct rds_ib_fastreg *frr; > + > + frr = rds_ib_rdma_get_fastreg(mr); > + if (!frr) > + return 0; > + if (IS_ERR(frr)) > + return PTR_ERR(frr); > + return __rds_ib_xmit_fastreg(conn, frr); > +} > + > void rds_ib_xmit_complete(struct rds_connection *conn) > { > struct rds_ib_connection *ic = conn->c_transport_data; > Index: build-2.6/net/rds/send.c > =================================================================== > --- build-2.6.orig/net/rds/send.c > +++ build-2.6/net/rds/send.c > @@ -84,6 +84,10 @@ void rds_send_reset(struct rds_connectio > list_for_each_entry_safe(rm, tmp, &conn->c_retrans, m_conn_item) { > set_bit(RDS_MSG_ACK_REQUIRED, &rm->m_flags); > set_bit(RDS_MSG_RETRANSMITTED, &rm->m_flags); > + /* If we were in the process of performing a fastreg > + * memory registration when the connection went down, > + * we have to retry it. */ > + clear_bit(RDS_MSG_FASTREG_POSTED, &rm->m_flags); > } > list_splice_init(&conn->c_retrans, &conn->c_send_queue); > spin_unlock_irqrestore(&conn->c_lock, flags); > @@ -765,6 +769,9 @@ static int rds_cmsg_send(struct rds_sock > if (cmsg->cmsg_level != SOL_RDS) > continue; > > + /* As a side effect, RDMA_DEST and RDMA_MAP will set > + * rm->m_rdma_cookie and rm->m_rdma_mr. > + */ > switch (cmsg->cmsg_type) { > case RDS_CMSG_RDMA_ARGS: > ret = rds_cmsg_rdma_args(rs, rm, cmsg); > Index: build-2.6/net/rds/message.c > =================================================================== > --- build-2.6.orig/net/rds/message.c > +++ build-2.6/net/rds/message.c > @@ -71,6 +71,8 @@ static void rds_message_purge(struct rds > > if (rm->m_rdma_op) > rds_rdma_free_op(rm->m_rdma_op); > + if (rm->m_rdma_mr) > + rds_mr_put(rm->m_rdma_mr); > } > > void rds_message_inc_purge(struct rds_incoming *inc) > Index: build-2.6/net/rds/rdma.h > =================================================================== > --- build-2.6.orig/net/rds/rdma.h > +++ build-2.6/net/rds/rdma.h > @@ -74,4 +74,11 @@ int rds_cmsg_rdma_map(struct rds_sock *r > void rds_rdma_free_op(struct rds_rdma_op *ro); > void rds_rdma_send_complete(struct rds_message *rm, int); > > +extern void __rds_put_mr_final(struct rds_mr *mr); > +static inline void rds_mr_put(struct rds_mr *mr) > +{ > + if (atomic_dec_and_test(&mr->r_refcount)) > + __rds_put_mr_final(mr); > +} > + > #endif > Index: build-2.6/net/rds/ib_recv.c > =================================================================== > --- build-2.6.orig/net/rds/ib_recv.c > +++ build-2.6/net/rds/ib_recv.c > @@ -796,7 +796,7 @@ void rds_ib_recv_cq_comp_handler(struct > while (ib_poll_cq(cq, 1, &wc) > 0) { > rdsdebug("wc wr_id 0x%llx status %u byte_len %u imm_data %u\n", > (unsigned long long)wc.wr_id, wc.status, wc.byte_len, > - be32_to_cpu(wc.imm_data)); > + be32_to_cpu(wc.ex.imm_data)); > rds_ib_stats_inc(s_ib_rx_cq_event); > > recv = &ic->i_recvs[rds_ib_ring_oldest(&ic->i_recv_ring)]; From olaf.kirch at oracle.com Mon Jul 7 10:14:19 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Mon, 7 Jul 2008 19:14:19 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080707165555.GB11451@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807071440.46502.olaf.kirch@oracle.com> <20080707165555.GB11451@opengridcomputing.com> Message-ID: <200807071914.19946.olaf.kirch@oracle.com> On Monday 07 July 2008 18:55:55 Jon Mason wrote: > It doesn't seem to work. After looking through the code, the > check_laddr will fail because it is still looking for the IB arp and not > the inet arp needed by iWARP. I hacked around that but it is still not > working. I'll look through the patch more and see if I can determine > what is still breaking. Okay, thanks for checking. I would really like to have a consolidated IB/iwarp transport with just minor differences being special-cased by checking for iWARP NICs and/or the new mem mgmt interface. Note that I did not change the port for iWARP. Would this cause trouble for your cards? (Yes, we probably need to use a separate port anyway, for cards that do both IB and iwarp - but I wanted to keep the patch minimal initially) Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From rdreier at cisco.com Mon Jul 7 10:56:17 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 10:56:17 -0700 Subject: [ofa-general] Re: [PATCH 1/4] rdma/addr: keep pointer to the netdevice in struct rdma_dev_addr In-Reply-To: <4871AF8B.7020207@voltaire.com> (Or Gerlitz's message of "Mon, 07 Jul 2008 08:54:19 +0300") References: <4871AF8B.7020207@voltaire.com> Message-ID: > > > Keep a pointer to the local (src) netdevice in struct rdma_dev_addr. In a > > > High-Availability scheme this information can be used by the rdma-cm to align RDMA > > > sessions to use the same links as the IP stack does under fail-over and route change cases. > > I don't understand how keeping the netdevice is related to this chunk: > The rdma-cm doesn't issue address resolution (route lookup, neigh > lookup, etc) in the passive side of the connection and hence there is > some asymmetry between the code path taken at the client side vs the > server side. This difference had two implications: > > - the passive side network device was not resolved, where now it is > resolved by rdma_translate_ip > > - there was some trivial code duplication between rdma_copy_addr to > cma_new_conn_id, which is now removed as with this patch the chain of > calls becomes cma_new_conn_id --> rdma_translate_ip --> rdma_copy_addr So you need to add the call to rdma_translate_ip() now to get the netdevice set properly? From swise at opengridcomputing.com Mon Jul 7 11:07:32 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 13:07:32 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> References: <20080703213412.GB32285@opengridcomputing.com> <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> Message-ID: <48725B64.7000904@opengridcomputing.com> >> +struct ib_device *ipaddr_to_ibdev(u32 addr) >> > > is there a reason why rdma_resolve_addr(NULL, addr, timeout) wouldn't > do what ipaddr_to_ibdev does? > > rdma_resolve_addr(NULL,...) doesn't work in the current code. Are you suggesting allowing rdma_resolve_addr() with a NULL cm_id? The reason for not using rdma_resolve_addr() is to avoid having to allocate and free a cm_id each time we just want to find the local ib device, given a remote IP addr... Steve. From jon at opengridcomputing.com Mon Jul 7 11:30:26 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Mon, 7 Jul 2008 13:30:26 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807071914.19946.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807071440.46502.olaf.kirch@oracle.com> <20080707165555.GB11451@opengridcomputing.com> <200807071914.19946.olaf.kirch@oracle.com> Message-ID: <20080707183026.GD11451@opengridcomputing.com> On Mon, Jul 07, 2008 at 07:14:19PM +0200, Olaf Kirch wrote: > On Monday 07 July 2008 18:55:55 Jon Mason wrote: > > It doesn't seem to work. After looking through the code, the > > check_laddr will fail because it is still looking for the IB arp and not > > the inet arp needed by iWARP. I hacked around that but it is still not > > working. I'll look through the patch more and see if I can determine > > what is still breaking. > > Okay, thanks for checking. I would really like to have a consolidated > IB/iwarp transport with just minor differences being special-cased > by checking for iWARP NICs and/or the new mem mgmt interface. I 100% agree, but was under the false impression that you wanted it seperate. I understand now, and will work off the patch you sent out. > Note that I did not change the port for iWARP. Would this cause > trouble for your cards? I do not believe this is a problem (famous last words). I will see if I can determine what is breaking it and create a patch. Thanks, Jon > (Yes, we probably need to use a separate port anyway, for cards that > do both IB and iwarp - but I wanted to keep the patch minimal initially) > > Olaf > -- > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From sean.hefty at intel.com Mon Jul 7 12:52:25 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 7 Jul 2008 12:52:25 -0700 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48725B64.7000904@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com><15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> Message-ID: <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> >The reason for not using rdma_resolve_addr() is to avoid having to >allocate and free a cm_id each time we just want to find the local ib >device, given a remote IP addr... Can someone explain why this needed? The local IB device is resolved as part of the connection establishment process. Why is an additional mechanism needed? Also, please look at ib_addr.h to see if one of those interfaces could be used instead. - Sean From knqiraedmzvx at mystical1.every1.net Mon Jul 7 14:03:47 2008 From: knqiraedmzvx at mystical1.every1.net (chasity) Date: Mon, 07 Jul 2008 13:03:47 -0800 Subject: [ofa-general] it`s chasity Message-ID: Hi My name is chasity. I found your email on that dating site. I also love sex on the side. I have a loving partner but he is working 16 hours a day and we have sex only once a week :( If you are interested and wanna see my pictures just email me at cchasity606 at hugdr.info Don`t reply, use the email above (my boyfriend doesn`t know about that email!) From swise at opengridcomputing.com Mon Jul 7 13:18:19 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 15:18:19 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> References: <20080703213412.GB32285@opengridcomputing.com><15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> Message-ID: <48727A0B.8050109@opengridcomputing.com> Sean Hefty wrote: >> The reason for not using rdma_resolve_addr() is to avoid having to >> allocate and free a cm_id each time we just want to find the local ib >> device, given a remote IP addr... >> > > Can someone explain why this needed? The local IB device is resolved as part of > the connection establishment process. Why is an additional mechanism needed? > > RDS experts, correct me if this is wrong: RDS exposes a reliable datagram service to user apps. To do this it keeps connections to each host with RDS apps running and muxes user datagrams over these connections using its own protocol. Further there are 3 types of "transports" supported: TCP/IP, IB, and IW. When a user app sends a datagram, the RDS module must figure out which transport should be used. It does this based on the destination (and maybe local?) ip addr of the RDS application's socket. Prior to the iWARP patch, the way this transport switch decision was made was to look up the local netdev based on the ipaddr(s) and if the netdev's ARP header type was INFINIBAND, then obviously the IB transport should be used. With the addition of iWARP, the decision is a little more complicated because the iWARP netdev arp type is ETHERNET. So Jon added this new cma service to find the netdev, then take the netdev mac address and search the gid cache to see if that mac address was actually a gid for an rdma device. If the mac addr was found, then we know this netdev supports rdma. Creating the new service simply made this lookup light-weight by not requiring a cm_id to be created, bound, etc. Maybe there's a better way? Steve. From sean.hefty at intel.com Mon Jul 7 13:47:58 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 7 Jul 2008 13:47:58 -0700 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48727A0B.8050109@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com><15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> Message-ID: <000901c8e072$bd404ae0$a3d4180a@amr.corp.intel.com> >Maybe there's a better way? I would look at using rdma_resolve_ip, or using rdma_resolve_addr as is. The rdma_cm_id ensures that there's a reference on the ib_device for the pointer that's returned. The proposed interface returns a pointer without any way to verify that it's still valid, or know when it is no longer usable. - Sean From robert.j.woodruff at intel.com Mon Jul 7 13:48:53 2008 From: robert.j.woodruff at intel.com (Woodruff, Robert J) Date: Mon, 7 Jul 2008 13:48:53 -0700 Subject: [ofa-general] Developer's Workshop at SC'08 Message-ID: Hi Guys, I have been asked by the OpenFabrics marketing working group to gauge the possible interest in having a developer workshop during the week of SC'08 in Austin. There are already plans to have BOFs at SC'08 to provide customers with updates on the upcoming releases, i.e, OFED 1.4 and WinOF 2.0. The question is, do you think we need a developers workshop that week as well to perhaps plan future work, OFED 1.5, WinOF 2.1 etc. or do you think a face-to-face meeting is not needed this fall and wait till next spring at Sonoma for the next face-to-face. Thoughts ? woody From John.Marshall at ec.gc.ca Mon Jul 7 13:51:18 2008 From: John.Marshall at ec.gc.ca (John Marshall) Date: Mon, 07 Jul 2008 20:51:18 +0000 Subject: [ofa-general] debian build instructions? Message-ID: <487281C6.5070001@ec.gc.ca> Hi, I have muddled my way through building the latest ofed-1.3.1 on debian etch. Is there a simple way of doing this that I am missing? I have searched for info and found some, but nothing that gives a step by step procedure. Thanks, John From hrosenstock at xsigo.com Mon Jul 7 14:07:31 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 07 Jul 2008 14:07:31 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] libibmad/src/rpc.c: Eliminate compiler warning Message-ID: <1215464852.18371.70.camel@hrosenstock-ws.xsigo.com> libibmad/src/rpc.c: Eliminate compiler warning Signed-off-by: Hal Rosenstock diff --git a/libibmad/src/rpc.c b/libibmad/src/rpc.c index a2cec95..df28f65 100644 --- a/libibmad/src/rpc.c +++ b/libibmad/src/rpc.c @@ -213,7 +213,7 @@ void * mad_rpc_rmpp(const void *port_id, ib_rpc_t *rpc, ib_portid_t *dport, ib_rmpp_hdr_t *rmpp, void *data) { - struct ibmad_port *p = port_id; + const struct ibmad_port *p = port_id; int status, len; uint8_t sndbuf[1024], rcvbuf[1024], *mad; From swise at opengridcomputing.com Mon Jul 7 14:10:09 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 16:10:09 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <000901c8e072$bd404ae0$a3d4180a@amr.corp.intel.com> References: <20080703213412.GB32285@opengridcomputing.com><15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> <000901c8e072$bd404ae0$a3d4180a@amr.corp.intel.com> Message-ID: <48728631.3010406@opengridcomputing.com> Sean Hefty wrote: >> Maybe there's a better way? >> > > I would look at using rdma_resolve_ip, or using rdma_resolve_addr as is. > > The rdma_cm_id ensures that there's a reference on the ib_device for the pointer > that's returned. The proposed interface returns a pointer without any way to > verify that it's still valid, or know when it is no longer usable. > > The rds module already _has_ cm_id's bound to these devices. So they won't "go away" without rds getting an event. The service in question was just a quick way to map an ip address to its associated struct ib_device. Perhaps this just needs to be a function inside the rds module as opposed to something exported from the cma proper... Steve. From hrosenstock at xsigo.com Mon Jul 7 14:10:25 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 07 Jul 2008 14:10:25 -0700 Subject: [ofa-general] [PATCH] libibmad: Initial changes for congestion management class Message-ID: <1215465025.18371.74.camel@hrosenstock-ws.xsigo.com> libibmad: Initial changes for congestion management class Signed-off-by: Hal Rosenstock diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h index b12cd6b..c2ad148 100644 --- a/libibmad/include/infiniband/mad.h +++ b/libibmad/include/infiniband/mad.h @@ -81,6 +81,7 @@ enum MAD_CLASSES { IB_SNMP_CLASS = 0x8, IB_VENDOR_RANGE1_START_CLASS = 0x9, IB_VENDOR_RANGE1_END_CLASS = 0x0f, + IB_CC_CLASS = 0x21, IB_VENDOR_RANGE2_START_CLASS = 0x30, IB_VENDOR_RANGE2_END_CLASS = 0x4f, }; diff --git a/libibmad/src/register.c b/libibmad/src/register.c index 1698f05..a33acd8 100644 --- a/libibmad/src/register.c +++ b/libibmad/src/register.c @@ -97,6 +97,8 @@ mgmt_class_vers(int mgmt_class) return 1; case IB_DEVICE_MGMT_CLASS: return 1; + case IB_CC_CLASS: + return 2; } return 0; From hrosenstock at xsigo.com Mon Jul 7 14:12:58 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 07 Jul 2008 14:12:58 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] infiniband-diags/mcm_rereg_test.c: Cosmetic formatting change Message-ID: <1215465178.18371.78.camel@hrosenstock-ws.xsigo.com> infiniband-diags/mcm_rereg_test.c: Cosmetic formatting change Signed-off-by: Hal Rosenstock diff --git a/infiniband-diags/src/mcm_rereg_test.c b/infiniband-diags/src/mcm_rereg_test.c index 2bdf9dc..9285b95 100644 --- a/infiniband-diags/src/mcm_rereg_test.c +++ b/infiniband-diags/src/mcm_rereg_test.c @@ -99,7 +99,6 @@ static void build_mcm_rec_umad(void *umad, ib_portid_t *dport, int method, rpc.datasz = IB_SA_DATA_SIZE; rpc.dataoffs = IB_SA_DATA_OFFS; - mad_build_pkt(umad, &rpc, dport, NULL, data); } From jon at opengridcomputing.com Mon Jul 7 14:13:21 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Mon, 7 Jul 2008 16:13:21 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48728631.3010406@opengridcomputing.com> References: <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> <000901c8e072$bd404ae0$a3d4180a@amr.corp.intel.com> <48728631.3010406@opengridcomputing.com> Message-ID: <20080707211321.GE11451@opengridcomputing.com> On Mon, Jul 07, 2008 at 04:10:09PM -0500, Steve Wise wrote: > Sean Hefty wrote: >>> Maybe there's a better way? >>> >> >> I would look at using rdma_resolve_ip, or using rdma_resolve_addr as is. >> >> The rdma_cm_id ensures that there's a reference on the ib_device for the pointer >> that's returned. The proposed interface returns a pointer without any way to >> verify that it's still valid, or know when it is no longer usable. >> >> > The rds module already _has_ cm_id's bound to these devices. So they > won't "go away" without rds getting an event. The service in question > was just a quick way to map an ip address to its associated struct > ib_device. Perhaps this just needs to be a function inside the rds > module as opposed to something exported from the cma proper... Unfortunately, this cannot be rds only. All the data structs are local to cma.c. The question is, if given only a u32 ipaddr, is there a way to turn this into a struct ib_device *? Thanks, Jon > > Steve. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From swise at opengridcomputing.com Mon Jul 7 14:14:20 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 16:14:20 -0500 Subject: [ofa-general] Re: [PATCH RFC] rds: add iwarp support In-Reply-To: <20080703213412.GB32285@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> Message-ID: <4872872C.7030006@opengridcomputing.com> > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index 0751697..0c5028a 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -36,6 +36,7 @@ > #include > #include > #include > +#include > > #include > > @@ -200,6 +201,43 @@ struct sdp_hah { > #define CMA_VERSION 0x00 > #define SDP_MAJ_VERSION 0x2 > > +struct ib_device *ipaddr_to_ibdev(u32 addr) > +{ > + struct rdma_dev_addr dev_addr; > + struct cma_device *cma_dev; > + struct net_device *dev; > + union ib_gid gid; > + > + dev = ip_dev_find(addr); > + if (!dev) > + goto out; > + > + rdma_copy_addr(&dev_addr, dev, NULL); > + > + switch (dev->type) { > + case ARPHRD_INFINIBAND: > + ib_addr_get_sgid(&dev_addr, &gid); > + break; > + case ARPHRD_ETHER: > + iw_addr_get_sgid(&dev_addr, &gid); > + break; > + default: > + goto out; > + } > + > + list_for_each_entry(cma_dev, &dev_list, list) { > + int ret; > + u8 port; > + > + ret = ib_find_cached_gid(cma_dev->device, &gid, &port, NULL); > + if (!ret) > + return cma_dev->device; > + } > +out: > + return NULL; > +} > +EXPORT_SYMBOL(ipaddr_to_ibdev); > + > static int cma_comp(struct rdma_id_private *id_priv, enum cma_state comp) > { > unsigned long flags; > Pending our discussion on whether this service will even survive, there is a bug in it. Function ip_dev_find() keeps a reference on the netdev. So before returning, this function should do a dev_put(dev) to release the reference... From swise at opengridcomputing.com Mon Jul 7 14:20:43 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 16:20:43 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080707211321.GE11451@opengridcomputing.com> References: <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> <000901c8e072$bd404ae0$a3d4180a@amr.corp.intel.com> <48728631.3010406@opengridcomputing.com> <20080707211321.GE11451@opengridcomputing.com> Message-ID: <487288AB.5040204@opengridcomputing.com> Jon Mason wrote: > On Mon, Jul 07, 2008 at 04:10:09PM -0500, Steve Wise wrote: > >> Sean Hefty wrote: >> >>>> Maybe there's a better way? >>>> >>>> >>> I would look at using rdma_resolve_ip, or using rdma_resolve_addr as is. >>> >>> The rdma_cm_id ensures that there's a reference on the ib_device for the pointer >>> that's returned. The proposed interface returns a pointer without any way to >>> verify that it's still valid, or know when it is no longer usable. >>> >>> >>> >> The rds module already _has_ cm_id's bound to these devices. So they >> won't "go away" without rds getting an event. The service in question >> was just a quick way to map an ip address to its associated struct >> ib_device. Perhaps this just needs to be a function inside the rds >> module as opposed to something exported from the cma proper... >> > > Unfortunately, this cannot be rds only. All the data structs are local > to cma.c. > > The question is, if given only a u32 ipaddr, is there a way to turn this > into a struct ib_device *? > > Thanks, > Jon > > Ah yes. The cma dev list is static. But ib_find_cached_gid() is exported. If you only really ever need to do this on active connections, then you could walk your list of connected cm_ids and get the struct ib_device for each and call ib_find_cached_gid() to see if its a match. Would that work? It would handle setting up new connections, but maybe that is a different path? steve. From chu11 at llnl.gov Mon Jul 7 14:25:52 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 07 Jul 2008 14:25:52 -0700 Subject: [ofa-general] [OPENSM] [PATCH] reroute console option Message-ID: <1215465953.16508.58.camel@cardanus.llnl.gov> Hey Sasha, With all of these file-input possibilities for OpenSM routing, I don't think there is a way to tell OpenSM to reroute if (for example) the file input has changed but the network/fabric has not. This is a patch to add a command to the console to tell the console to reroute but not do a heavy sweep. Longer term, perhaps there could be a plugin/thread/something to watch the routing input files and see if they change, then set the "force_reroute" flag as needed. But I think this console option should suffice for now. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-reroute-console-command.patch Type: text/x-patch Size: 4501 bytes Desc: not available URL: From rdreier at cisco.com Mon Jul 7 14:30:27 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 14:30:27 -0700 Subject: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <200807061300.31811.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Sun, 6 Jul 2008 13:00:31 +0300") References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> <200807061300.31811.jackm@dev.mellanox.co.il> Message-ID: > Probably, the least short-term pain (no ABI change) would be to use the last reserved > create-qp ABI byte for flags, while defining the flags field in > ibv_qp_init_attr as uint32_t (preparing for more than 8 future flags). Actually there are tricks we could do to get more flag bits in the kernel ABI... eg set a bit in the reserved part to indicate "flags are present", and then stick the flags in srq_handle if is_srq is not set, and in max_recv_sge if it is set. > Adding the flags field to the qp_init_attr structure does, however, require incrementing the > libibverbs version to 1.2, necessitating an IBVERBS_1.1 compatibility layer. This actually seems like much more of an issue. Especially since librdmacm calls ibv_create_qp. - R. From rdreier at cisco.com Mon Jul 7 14:31:20 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 14:31:20 -0700 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <3b5e77ad0807070220i9f8659cm4f6e92e0286fcf5@mail.gmail.com> (Ron Livne's message of "Mon, 7 Jul 2008 12:20:47 +0300") References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> <200807061300.31811.jackm@dev.mellanox.co.il> <3b5e77ad0807070220i9f8659cm4f6e92e0286fcf5@mail.gmail.com> Message-ID: > What do you think of the following approach? > Instead of adding creation flags to the qp_init_attr, I can add a new verb: > ibv_qp *create_qp_extended(struct ibv_pd *pd, struct ibv_qp_init_attr, > *init_attr, enum ibv_qp_create_flags create_flags) > > I'm aware that adding a new verb isn't optimal, but at least we can > avoid incrementing the libibverbs version. I think this new verb seems like a better approach right now. - R. From olaf.kirch at oracle.com Mon Jul 7 14:32:05 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Mon, 7 Jul 2008 23:32:05 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48727A0B.8050109@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> Message-ID: <200807072332.06252.olaf.kirch@oracle.com> On Monday 07 July 2008 22:18:19 Steve Wise wrote: > Sean Hefty wrote: > >> The reason for not using rdma_resolve_addr() is to avoid having to > >> allocate and free a cm_id each time we just want to find the local ib > >> device, given a remote IP addr... > >> > > > > Can someone explain why this needed? The local IB device is resolved as part of > > the connection establishment process. Why is an additional mechanism needed? > > > > > > RDS experts, correct me if this is wrong: Essentially, you bind an RDS socket to an IP address, and that tells RDS to pick a transport. That's what laddr_check is for - it's called from bind() to determine what kind of transport to use. Olaf > > RDS exposes a reliable datagram service to user apps. To do this it > keeps connections to each host with RDS apps running and muxes user > datagrams over these connections using its own protocol. Further there > are 3 types of "transports" supported: TCP/IP, IB, and IW. When a user > app sends a datagram, the RDS module must figure out which transport > should be used. It does this based on the destination (and maybe local?) > ip addr of the RDS application's socket. Prior to the iWARP patch, the > way this transport switch decision was made was to look up the local > netdev based on the ipaddr(s) and if the netdev's ARP header type was > INFINIBAND, then obviously the IB transport should be used. With the > addition of iWARP, the decision is a little more complicated because the > iWARP netdev arp type is ETHERNET. So Jon added this new cma service to > find the netdev, then take the netdev mac address and search the gid > cache to see if that mac address was actually a gid for an rdma device. > If the mac addr was found, then we know this netdev supports rdma. > Creating the new service simply made this lookup light-weight by not > requiring a cm_id to be created, bound, etc. > > Maybe there's a better way? > > Steve. > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From swise at opengridcomputing.com Mon Jul 7 14:37:45 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 16:37:45 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807072332.06252.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> <200807072332.06252.olaf.kirch@oracle.com> Message-ID: <48728CA9.7070102@opengridcomputing.com> Olaf Kirch wrote: > On Monday 07 July 2008 22:18:19 Steve Wise wrote: > >> Sean Hefty wrote: >> >>>> The reason for not using rdma_resolve_addr() is to avoid having to >>>> allocate and free a cm_id each time we just want to find the local ib >>>> device, given a remote IP addr... >>>> >>>> >>> Can someone explain why this needed? The local IB device is resolved as part of >>> the connection establishment process. Why is an additional mechanism needed? >>> >>> >>> >> RDS experts, correct me if this is wrong: >> > > Essentially, you bind an RDS socket to an IP address, and that tells > RDS to pick a transport. That's what laddr_check is for - it's called > from bind() to determine what kind of transport to use. > > Olaf > > So that sounds like it is done once per RDS socket, not once per datagram send. Is this true? From swise at opengridcomputing.com Mon Jul 7 15:18:17 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 17:18:17 -0500 Subject: [ofa-general] regression Message-ID: <48729629.2090501@opengridcomputing.com> Hey Roland, I just noticed that the change it iwch_provider.c in commit f4e91eb4a81559da87a3843758a641b5cc590b65 undid the fix done in commit 7f049f2f4259e4a5c213720df1ce0c2ca095641a. It removed the calls to rtnl_lock() that serialized the iw_cxgb3 ethtool ops calls into the cxgb3 driver. Shall I add a patch to reintroduce the fix? Stevo. From rdreier at cisco.com Mon Jul 7 15:42:13 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 15:42:13 -0700 Subject: [ofa-general] Re: regression In-Reply-To: <48729629.2090501@opengridcomputing.com> (Steve Wise's message of "Mon, 07 Jul 2008 17:18:17 -0500") References: <48729629.2090501@opengridcomputing.com> Message-ID: > I just noticed that the change it iwch_provider.c in commit > f4e91eb4a81559da87a3843758a641b5cc590b65 ["IB: convert struct > class_device to struct device"] undid the fix done in commit > 7f049f2f4259e4a5c213720df1ce0c2ca095641a ["RDMA/cxgb3: Hold > rtnl_lock() around ethtool get_drvinfo call"]. It removed the calls > to rtnl_lock() that serialized the iw_cxgb3 ethtool ops calls into > the cxgb3 driver. > > Shall I add a patch to reintroduce the fix? Yeah, definitely... I can't see any reason why the rtnl_lock would be safe to remove. - R. From rdreier at cisco.com Mon Jul 7 15:46:34 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 15:46:34 -0700 Subject: [ofa-general] Re: regression In-Reply-To: (Roland Dreier's message of "Mon, 07 Jul 2008 15:42:13 -0700") References: <48729629.2090501@opengridcomputing.com> Message-ID: > > I just noticed that the change it iwch_provider.c in commit > > f4e91eb4a81559da87a3843758a641b5cc590b65 ["IB: convert struct > > class_device to struct device"] undid the fix done in commit > > 7f049f2f4259e4a5c213720df1ce0c2ca095641a ["RDMA/cxgb3: Hold > > rtnl_lock() around ethtool get_drvinfo call"]. It removed the calls > > to rtnl_lock() that serialized the iw_cxgb3 ethtool ops calls into > > the cxgb3 driver. > > > > Shall I add a patch to reintroduce the fix? > > Yeah, definitely... I can't see any reason why the rtnl_lock would be > safe to remove. although it's not clear to me that it's kosher for the cxgb3 get_drvinfo ethtool op to require serialization... From swise at opengridcomputing.com Mon Jul 7 16:08:04 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 07 Jul 2008 18:08:04 -0500 Subject: [ofa-general] Re: regression In-Reply-To: References: <48729629.2090501@opengridcomputing.com> Message-ID: <4872A1D4.8010100@opengridcomputing.com> Roland Dreier wrote: > > > I just noticed that the change it iwch_provider.c in commit > > > f4e91eb4a81559da87a3843758a641b5cc590b65 ["IB: convert struct > > > class_device to struct device"] undid the fix done in commit > > > 7f049f2f4259e4a5c213720df1ce0c2ca095641a ["RDMA/cxgb3: Hold > > > rtnl_lock() around ethtool get_drvinfo call"]. It removed the calls > > > to rtnl_lock() that serialized the iw_cxgb3 ethtool ops calls into > > > the cxgb3 driver. > > > > > > Shall I add a patch to reintroduce the fix? > > > > Yeah, definitely... I can't see any reason why the rtnl_lock would be > > safe to remove. > > although it's not clear to me that it's kosher for the cxgb3 get_drvinfo > ethtool op to require serialization... > ethtool ops assume the rtnl_lock is held. From net/core/dev.c: case SIOCETHTOOL: dev_load(net, ifr.ifr_name); rtnl_lock(); ret = dev_ethtool(net, &ifr); rtnl_unlock(); I'm just avoiding the overhead of building the ifr struct and calling the ioctl... From rdreier at cisco.com Mon Jul 7 16:11:37 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 07 Jul 2008 16:11:37 -0700 Subject: [ofa-general] Re: regression In-Reply-To: <4872A1D4.8010100@opengridcomputing.com> (Steve Wise's message of "Mon, 07 Jul 2008 18:08:04 -0500") References: <48729629.2090501@opengridcomputing.com> <4872A1D4.8010100@opengridcomputing.com> Message-ID: > ethtool ops assume the rtnl_lock is held. From net/core/dev.c: > > case SIOCETHTOOL: > dev_load(net, ifr.ifr_name); > rtnl_lock(); > ret = dev_ethtool(net, &ifr); > rtnl_unlock(); > > > > I'm just avoiding the overhead of building the ifr struct and calling > the ioctl... Yes, but cf the argument I'm having with davem on netdev right now about whether the get_drvinfo ethtool method is safe to call without rtnl... - R. From olaf.kirch at oracle.com Mon Jul 7 22:46:42 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Tue, 8 Jul 2008 07:46:42 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48728CA9.7070102@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807072332.06252.olaf.kirch@oracle.com> <48728CA9.7070102@opengridcomputing.com> Message-ID: <200807080746.43924.olaf.kirch@oracle.com> On Monday 07 July 2008 23:37:45 Steve Wise wrote: > > Essentially, you bind an RDS socket to an IP address, and that tells > > RDS to pick a transport. That's what laddr_check is for - it's called > > from bind() to determine what kind of transport to use. > > So that sounds like it is done once per RDS socket, not once per > datagram send. Is this true? Yes, correct. The socket/transport binding is established once, and never changes. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From jackm at dev.mellanox.co.il Mon Jul 7 23:23:02 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 8 Jul 2008 09:23:02 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: References: <3b5e77ad0807070220i9f8659cm4f6e92e0286fcf5@mail.gmail.com> Message-ID: <200807080923.03147.jackm@dev.mellanox.co.il> On Tuesday 08 July 2008 00:31, Roland Dreier wrote: > > What do you think of the following approach? > > Instead of adding creation flags to the qp_init_attr, I can add a new verb: > > ibv_qp *create_qp_extended(struct ibv_pd *pd, struct ibv_qp_init_attr, > > *init_attr, enum ibv_qp_create_flags create_flags) > > > > I'm aware that adding a new verb isn't optimal, but at least we can > > avoid incrementing the libibverbs version. > > I think this new verb seems like a better approach right now. can we do this so that the new verb lies after XRC in ib_context, since OFED 1.3 went out with XRC in userspace (so we don't break binary compatibility): struct ibv_context { struct ibv_device *device; struct ibv_context_ops ops; int cmd_fd; int async_fd; int num_comp_vectors; pthread_mutex_t mutex; void *abi_compat; struct ibv_xrc_ops *xrc_ops; /* ==> new verb ops structure pointer here */ }; Alternatively, we can rename ibv_xrc_ops to ibv_more_ops, and add the new verbs after the XRC operations (this is my preference, actually, so that we don't get into adding more and more structs at the end of the ibv_context structure): struct ibv_more_ops { struct ibv_srq * (*create_xrc_srq)(struct ibv_pd *pd, struct ibv_xrc_domain *xrc_domain, struct ibv_cq *xrc_cq, struct ibv_srq_init_attr *srq_init_attr); struct ibv_xrc_domain * (*open_xrc_domain)(struct ibv_context *context, int fd, int oflag); int (*close_xrc_domain)(struct ibv_xrc_domain *d); int (*create_xrc_rcv_qp)(struct ibv_qp_init_attr *init_attr, uint32_t *xrc_qp_num); int (*modify_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num, struct ibv_qp_attr *attr, int attr_mask); int (*query_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num, struct ibv_qp_attr *attr, int attr_mask, struct ibv_qp_init_attr *init_attr); int (*reg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int (*unreg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); struct ibv_qp * (*create_qp_expanded)(struct ibv_pd *pd, struct ibv_qp_init_attr *attr, uint32_t flags); }; struct ibv_context { struct ibv_device *device; struct ibv_context_ops ops; int cmd_fd; int async_fd; int num_comp_vectors; pthread_mutex_t mutex; void *abi_compat; struct ibv_more_ops *more_ops; }; BTW, I suggest calling the new verb "ibv_create_qp_expanded()", so as not to confuse things with XRC (Extended RC). - Jack From vlad at lists.openfabrics.org Tue Jul 8 02:45:28 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 8 Jul 2008 02:45:28 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080708-0200 daily build status Message-ID: <20080708094528.F1929E6096A@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080708-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Tue Jul 8 02:52:34 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 12:52:34 +0300 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <487288AB.5040204@opengridcomputing.com> References: <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> <000901c8e072$bd404ae0$a3d4180a@amr.corp.intel.com> <48728631.3010406@opengridcomputing.com> <20080707211321.GE11451@opengridcomputing.com> <487288AB.5040204@opengridcomputing.com> Message-ID: <487338E2.8050909@voltaire.com> >> Unfortunately, this cannot be rds only. All the data structs are >> local to cma.c. The question is, if given only a u32 ipaddr, is >> there a way to turn this into a struct ib_device *? > Ah yes. The cma dev list is static. But ib_find_cached_gid() is > exported. If you only really ever need to do this on active > connections, then you could walk your list of connected cm_ids and get > the struct ib_device for each and call ib_find_cached_gid() to see if > its a match. Would that work? It would handle setting up new > connections, but maybe that is a different path? > > collecting the suggestions made here on this relative small issue, I see bunch of problems: 1. u32 addr means IPv4 where an effort has been made to design the rdma-cm such that it supports both IPv4 and IPv6. The latter isn't complete yet, but it just a matter of internal implementation. Same goes to the RDMA ULPs, I think we should avoid doing things which are special to IPv4 or better, not assume IPv4 in the ULP code, as long as it does not go with much pain. 2. Roland commented few times over the last months that the ib_find_cached_xxx calls are to be removed, so lets not use them here. > Essentially, you bind an RDS socket to an IP address, and that tells RDS to pick a transport. That's what laddr_check is for - it's called from bind() to determine what kind of transport to use. 3. Olaf has commented that indeed, there is no reason notable at this point to have IB and iWARP use different transports, so lets not design a mechanism which is not needed at all. Or From ogerlitz at voltaire.com Tue Jul 8 03:01:28 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 13:01:28 +0300 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48727A0B.8050109@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com><15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> Message-ID: <48733AF8.3030208@voltaire.com> Steve Wise wrote: > Prior to the iWARP patch, the way this transport switch decision was > made was to look up the local netdev based on the ipaddr(s) and if the > netdev's ARP header type was INFINIBAND, then obviously the IB > transport should be used. With the addition of iWARP, the decision is > a little more complicated because the iWARP netdev arp type is ETHERNET. Olaf, Since the address provided to laddr_check() is the --source-- one (please correct me if I am wrong) and this is slow path operation, how about changing the transport switch to use rdma_bind_addr instead of ip_dev_find()? rdma_bind_addr is a synchronous operation, works for both IB and iWARP, and down the road will also support IPv6 (the API is in place) Or. From ogerlitz at voltaire.com Tue Jul 8 03:03:34 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 13:03:34 +0300 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48725B64.7000904@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> Message-ID: <48733B76.9060400@voltaire.com> Steve Wise wrote: > The reason for not using rdma_resolve_addr() is to avoid having to > allocate and free a cm_id each time we just want to find the local ib > device, given a remote IP addr... > its slow path op and hence acceptable to my taste, as I have elaborated on the other post. Or. From ogerlitz at voltaire.com Tue Jul 8 03:11:40 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 13:11:40 +0300 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807062138.29103.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807041207.40490.olaf.kirch@oracle.com> <200807041346.52282.olaf.kirch@oracle.com> <200807062138.29103.olaf.kirch@oracle.com> Message-ID: <48733D5C.7060000@voltaire.com> Olaf Kirch wrote: > Okay, there's a third way to fix this, which may even work with today's Oracle IPC code. The hack is to not submit the fastreg WR until we post a SEND that transmits the rkey to a peer. Then > > - we have a QP to post the WRs to > - we post the fastreg and the SEND to the same QP, > ensuring that the peer doesn't initiate the RDMA > before we've set up the memory registration. Yes, as Steve noted, using the same QP for the fastreg and send that carries the rkey/stag is a must since the send should be fenced on the fastreg completion to make sure the remote side uses a valid key. Or. From ogerlitz at voltaire.com Tue Jul 8 03:29:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 13:29:38 +0300 Subject: [ofa-general] Re: [PATCH RFC] rds: add iwarp support In-Reply-To: <200807040009.47794.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807040009.47794.olaf.kirch@oracle.com> Message-ID: <48734192.90604@voltaire.com> Olaf Kirch wrote: > As Or already noted, it would be preferable to have IB and iWARP share as much as possible. I understand there probably needs to be a different transport class for iWARP, but I would like iWARP to be some kind of "we do everything like IB except these two little bits" transport. Just to clarify, the new "fastreg as work requests" API is NOT iWARP specific, but rather comply in 95% to both IB and iWARP specs. Currently, support for these verbs has been posted only for the Chelsio driver (to be merged in 2.6.27-rc1), but Mellanox has made a comment over the list that support for the mlx4 is planned. So the distinction is between devices that 1. don't support any sort of fastreg 2. support the proprietary fmr verbs 3. support the standard fastreg as work request verbs my understanding is that RDS does not support type one ones, and its a question whether maintainance wise, you want to support type two ones. Other then that fastreg issue, and assuming the credit management issue is solved (is it?) is there any difference between IB to iWARP which you see as relevant to RDS v3 (p2p + rdma)? Or. The 5% difference are things supported by iWARP but not by IB such as read-with-invalidate, etc From ogerlitz at voltaire.com Tue Jul 8 04:02:58 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 14:02:58 +0300 Subject: [ofa-general] Getting Started with RDMA In-Reply-To: <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> Message-ID: <48734962.8000401@voltaire.com> Adrien Guillon wrote: > I'll check out the man pages today. I was hoping for something that > would give me a broad overview of the library implementation, the > approach it takes and assumptions made. I'm a complete n00b with > RDMA, but I would like to design a good C++ library to wrap the C > implementation.... unless there is such a thing already. There was some posting in the past on C++ object library that wraps libibverbs, here it is http://lists.openfabrics.org/pipermail/general/2008-May/050190.html > > Is this forum appropriate for asking n00b questions as I get started > programming with RDMA? basically yes, however I think it would be more efficient for you to take a look on the libibverbs and librdmacm man pages along with diving into some code examples and doing some runs. Specifically, the rping example provided by librdmacm-utils uses both libibverbs and librdmacm for the connected (RC) service (there's alos udaddy and mckey for datagrams), I suggest you start from there. Please note that you have to have a netdevice up and running for the IP subnet addresses you would be using with the rdma-cm. In the IB case, this means IPoIB and in the iWARP case, it means a network device set by the iWARP HW provider, in the current implementations, it means Gb Ethernet devices. Or. From olaf.kirch at oracle.com Tue Jul 8 04:50:19 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Tue, 8 Jul 2008 13:50:19 +0200 Subject: [ofa-general] Re: [PATCH RFC] rds: add iwarp support In-Reply-To: <48734192.90604@voltaire.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807040009.47794.olaf.kirch@oracle.com> <48734192.90604@voltaire.com> Message-ID: <200807081350.21409.olaf.kirch@oracle.com> On Tuesday 08 July 2008 12:29:38 Or Gerlitz wrote: > Olaf Kirch wrote: > > As Or already noted, it would be preferable to have IB and iWARP share as much as possible. > > I understand there probably needs to be a different transport class for iWARP, but I would > > like iWARP to be some kind of "we do everything like IB except these two little bits" transport. > > Just to clarify, the new "fastreg as work requests" API is NOT iWARP > specific, but rather comply in 95% to both IB and iWARP specs. Yes, that much I understand. For that reason, the rds_ib_device has two new flags: is_iwarp (local end point is iwarp device), and use_fastreg (device supports the fastreg interface, so create a mr_pool that uses fastreg instead of fmrs) > Currently, support for these verbs has been posted only for the Chelsio > driver (to be merged in 2.6.27-rc1), but Mellanox has made a comment > over the list that support for the mlx4 is planned. > > So the distinction is between devices that > > 1. don't support any sort of fastreg > 2. support the proprietary fmr verbs > 3. support the standard fastreg as work request verbs > > my understanding is that RDS does not support type one ones, and its a > question whether maintainance wise, you want to support type two ones. Indeed, choice #1 isn't supported. The patch I posted will pick fastreg whenever the device advertises it. I would have preferred to do it the other way round, because the fmr code is pretty well tested by now. Unfortunately, there doesn't seem to be a capability bit that says "I support the fmr interface", and I didn't really want to start peeking at the function pointers. > Other then that fastreg issue, and assuming the credit management issue > is solved (is it?) is there any difference between IB to iWARP which you > see as relevant to RDS v3 (p2p + rdma)? Yes. credit management is there in 1.4, and on by default. I have to say I have no clue what other differences between ib and iwarp may be relevant. I can't think of any, but that's probably just my lack of imagination. > The 5% difference are things supported by iWARP but not by IB such as > read-with-invalidate, etc Yeah. Let's get it working first, then > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From olaf.kirch at oracle.com Tue Jul 8 05:00:16 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Tue, 8 Jul 2008 14:00:16 +0200 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48733D5C.7060000@voltaire.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807062138.29103.olaf.kirch@oracle.com> <48733D5C.7060000@voltaire.com> Message-ID: <200807081400.17025.olaf.kirch@oracle.com> On Tuesday 08 July 2008 12:11:40 Or Gerlitz wrote: > Olaf Kirch wrote: > > Okay, there's a third way to fix this, which may even work with today's Oracle IPC code. The hack is to not submit the fastreg WR until we post a SEND that transmits the rkey to a peer. Then > > > > - we have a QP to post the WRs to > > - we post the fastreg and the SEND to the same QP, > > ensuring that the peer doesn't initiate the RDMA > > before we've set up the memory registration. > Yes, as Steve noted, using the same QP for the fastreg and send that > carries the rkey/stag is a must since the send should be fenced on the > fastreg completion to make sure the remote side uses a valid key. One theoretical problem is what happens if the application wants to register a memory range and send the rkey to two peers at the same time? We would have to stall the second send explicitly as long as we haven't seen the completion for the first send. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From hbisa at us.ibm.com Tue Jul 8 05:11:16 2008 From: hbisa at us.ibm.com (Hakem B Isa) Date: Tue, 8 Jul 2008 08:11:16 -0400 Subject: [ofa-general] Hakem B Isa/Cranford/IBM is out of the office. Message-ID: I will be out of the office starting 07/08/2008 and will not return until 07/14/2008. I will be out of the office If you need immediate help, please call my cell # Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Tue Jul 8 05:52:32 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 15:52:32 +0300 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807081400.17025.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807062138.29103.olaf.kirch@oracle.com> <48733D5C.7060000@voltaire.com> <200807081400.17025.olaf.kirch@oracle.com> Message-ID: <48736310.4080403@voltaire.com> Olaf Kirch wrote: > One theoretical problem is what happens if the application wants to register a memory range and send the rkey to two peers at the same time? We would have to stall the second send explicitly as long as we haven't seen the completion for the first send. Indeed. Basically I think that the fastreg designers envisioned a p2p (i.e client - server) transactional protocol where the client flow is: c1. get bunch of pages from upper layer c2. apply dma mapping c3. get MR from a preallocated pool, compute new rkey (8/32 bits portion) for this MR c4. issue fastreg WR on the pages to make the new rkey valid for this MR c5. issue send "request for rdma transaction" which contains the new rkey/stag second the server does s1: process request, get rkey, get local buffer from pool s2: issue rdma WR s3: write the data to backend storage or deliver it to upper layers upper layers s4: send response + remote invalidate WR on the rdma completion third the client does c6: recv response c7: return MR to pool c8: apply dma unmapping c9: hand the response to upper layers note that - stage s1 can be further enhanced to use fastreg also at the server side - there might be some ack to get from storage / upper layers between s3 to s4 - you probably want to negotiate the remote invalidate capability between the client and server upon connection set up Do you have an applicative usage for registering the same --memory-- with the same --rkey-- for two peers? Or. From ogerlitz at voltaire.com Tue Jul 8 05:57:45 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 15:57:45 +0300 Subject: [ofa-general] Re: [PATCH RFC] rds: add iwarp support In-Reply-To: <200807081350.21409.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807040009.47794.olaf.kirch@oracle.com> <48734192.90604@voltaire.com> <200807081350.21409.olaf.kirch@oracle.com> Message-ID: <48736449.6020800@voltaire.com> Olaf Kirch wrote: > Yes, that much I understand. For that reason, the rds_ib_device has two new flags: is_iwarp (local end point is iwarp device), and use_fastreg (device supports the fastreg interface, so create a mr_pool that uses fastreg instead of fmrs) OK, got it. I am not sure why you need the "is_iwarp" flag, guess I have to look deeper into the patch you have posted or maybe you can comment on that here... > Indeed, choice #1 isn't supported. The patch I posted will pick fastreg whenever > the device advertises it. I would have preferred to do it the other way round, > because the fmr code is pretty well tested by now. sounds fine for this point of time. > Yes. credit management is there in 1.4, and on by default. cool. > > I have to say I have no clue what other differences between ib and iwarp may be relevant. I can't think of any, but that's probably just my lack of imagination. I suggest to use the optimistic set of mind for this and learn as we move forward. >> The 5% difference are things supported by iWARP but not by IB such read-with-invalidate, etc > Yeah. Let's get it working first, then sure. Or. From rajib.majumder at credit-suisse.com Tue Jul 8 06:06:53 2008 From: rajib.majumder at credit-suisse.com (Majumder, Rajib) Date: Tue, 8 Jul 2008 21:06:53 +0800 Subject: [ofa-general] Getting Started with RDMA In-Reply-To: <48734962.8000401@voltaire.com> Message-ID: <0175FAC12977B047809C1BACA25881AD02A8C8B2@ESNG17P32002A.csfb.cs-group.com> Hi, There's already a C++ wrapper facade library (libibvpp) that wraps libibverbs calls. Pls check at http://opensource.arc.nasa.gov/project/libibvpp Thanks -----Original Message----- From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Or Gerlitz Sent: Tuesday, July 08, 2008 7:03 PM To: Adrien Guillon Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] Getting Started with RDMA Adrien Guillon wrote: > I'll check out the man pages today. I was hoping for something that > would give me a broad overview of the library implementation, the > approach it takes and assumptions made. I'm a complete n00b with > RDMA, but I would like to design a good C++ library to wrap the C > implementation.... unless there is such a thing already. There was some posting in the past on C++ object library that wraps libibverbs, here it is http://lists.openfabrics.org/pipermail/general/2008-May/050190.html > > Is this forum appropriate for asking n00b questions as I get started > programming with RDMA? basically yes, however I think it would be more efficient for you to take a look on the libibverbs and librdmacm man pages along with diving into some code examples and doing some runs. Specifically, the rping example provided by librdmacm-utils uses both libibverbs and librdmacm for the connected (RC) service (there's alos udaddy and mckey for datagrams), I suggest you start from there. Please note that you have to have a netdevice up and running for the IP subnet addresses you would be using with the rdma-cm. In the IB case, this means IPoIB and in the iWARP case, it means a network device set by the iWARP HW provider, in the current implementations, it means Gb Ethernet devices. Or. _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ============================================================================== From ogerlitz at voltaire.com Tue Jul 8 06:33:13 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 08 Jul 2008 16:33:13 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: References: <15ddcffd0807031504n2d25c4b3y16d255dea6f1e427@mail.gmail.com> <200807061300.31811.jackm@dev.mellanox.co.il> <3b5e77ad0807070220i9f8659cm4f6e92e0286fcf5@mail.gmail.com> Message-ID: <48736C99.4090308@voltaire.com> Roland Dreier wrote: > > What do you think of the following approach? > > Instead of adding creation flags to the qp_init_attr, I can add a new verb: > > ibv_qp *create_qp_extended(struct ibv_pd *pd, struct ibv_qp_init_attr, > > *init_attr, enum ibv_qp_create_flags create_flags) > > > > I'm aware that adding a new verb isn't optimal, but at least we can > > avoid incrementing the libibverbs version. > > I think this new verb seems like a better approach right now. Sean, Assuming this solution (a create-qp-extended verb that gets also creation flags) would be the one implemented at libibverbs, are you fine with applying the same approach to librdmacm that is adding a matching call eg rdma-create-qp-extended that just calls the new verb? Also when are you planning the next release of librdmacm? Or. From swise at opengridcomputing.com Tue Jul 8 06:45:05 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 08:45:05 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48733AF8.3030208@voltaire.com> References: <20080703213412.GB32285@opengridcomputing.com><15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <000501c8e06a$fab1ca50$a3d4180a@amr.corp.intel.com> <48727A0B.8050109@opengridcomputing.com> <48733AF8.3030208@voltaire.com> Message-ID: <48736F61.7010607@opengridcomputing.com> Or Gerlitz wrote: > Steve Wise wrote: >> Prior to the iWARP patch, the way this transport switch decision was >> made was to look up the local netdev based on the ipaddr(s) and if >> the netdev's ARP header type was INFINIBAND, then obviously the IB >> transport should be used. With the addition of iWARP, the decision is >> a little more complicated because the iWARP netdev arp type is ETHERNET. > Olaf, > > Since the address provided to laddr_check() is the --source-- one > (please correct me if I am wrong) and this is slow path operation, how > about changing the transport switch to use rdma_bind_addr instead of > ip_dev_find()? rdma_bind_addr is a synchronous operation, works for > both IB and iWARP, and down the road will also support IPv6 (the API > is in place) > > Or. > Yes, after realizing this path is done only once when an RDS socket is bound, then we should probably just use rdma_bind_addr()... From swise at opengridcomputing.com Tue Jul 8 06:45:25 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 08:45:25 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48733B76.9060400@voltaire.com> References: <20080703213412.GB32285@opengridcomputing.com> <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <48733B76.9060400@voltaire.com> Message-ID: <48736F75.9010105@opengridcomputing.com> Or Gerlitz wrote: > Steve Wise wrote: >> The reason for not using rdma_resolve_addr() is to avoid having to >> allocate and free a cm_id each time we just want to find the local ib >> device, given a remote IP addr... >> > its slow path op and hence acceptable to my taste, as I have > elaborated on the other post. > > Or. I agree now that I understand. :) From swise at opengridcomputing.com Tue Jul 8 07:49:59 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 09:49:59 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <48733B76.9060400@voltaire.com> References: <20080703213412.GB32285@opengridcomputing.com> <15ddcffd0807031459l73c9e073x40e40126b394e01a@mail.gmail.com> <48725B64.7000904@opengridcomputing.com> <48733B76.9060400@voltaire.com> Message-ID: <48737E97.3030503@opengridcomputing.com> Just for the record, there are a few coding todos remaining once we get the main merged IB/IW transport: 1) Make use of the local_dma_lkey if the device supports it. This lkey can be used in the SGL entries for SEND, RECV, and the source of a WRITE. 2) Get rid of the dma_mr usage for the sink of an RDMA READ. For iWARP, the sink of a read requires REMOTE_WRITE access, and we do not want to expose all of memory for remote I/O. The transport will have to use a fastreg mr for these. This is potentially iwarp-only logic since IB allows an lkey for the sink of a READ. Implementing both of these will allow the iwarp logic to not depend on dma mrs at all. This is desirable for the cxgb3 device which has limitations on dma mrs. Steve. From olga.shern at gmail.com Tue Jul 8 08:27:02 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Tue, 8 Jul 2008 18:27:02 +0300 Subject: [ofa-general] Multicast traffic generates Bad P_Key trap in SM when working in partial member setup In-Reply-To: <1214509372.13056.645.camel@hrosenstock-ws.xsigo.com> References: <39C75744D164D948A170E9792AF8E7CA013E6E2B@exil.voltaire.com> <1213266626.14047.419.camel@hrosenstock-ws.xsigo.com> <1213270788.14047.451.camel@hrosenstock-ws.xsigo.com> <1213278134.14047.498.camel@hrosenstock-ws.xsigo.com> <1214509372.13056.645.camel@hrosenstock-ws.xsigo.com> Message-ID: Hi, I have tested the same scenario with OpenSM. What I have expected to see is trap 259 because of switch enforcement that is done by OpenSM. I don't see this trap, therefore I assume that ANAFA doesn't generate this trap (this is not mandatory only optional). I only see trap 257 that is generated by the node (with partial pkey) that sends the multicast traffic. The conclusion is that if there is switch enforcement the issue is not severe and we can live with it :) Olga On 6/26/08, Hal Rosenstock wrote: > On Thu, 2008-06-12 at 16:44 +0300, Olga Shern (Voltaire) wrote: > > Would you try this with OpenSM (and validate your theory about > > getting > > switch bad PKey traps v. end port bad PKey traps) or does VSM > > have such > > a mode (ingress/egress partition filtering) ? > > > > -- Hal > > > > Yes, I will test it with OpenSM > > Any update on this ? Thanks. > > -- Hal > > From hrosenstock at xsigo.com Tue Jul 8 09:05:44 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 08 Jul 2008 09:05:44 -0700 Subject: [ofa-general] Multicast traffic generates Bad P_Key trap in SM when working in partial member setup In-Reply-To: References: <39C75744D164D948A170E9792AF8E7CA013E6E2B@exil.voltaire.com> <1213266626.14047.419.camel@hrosenstock-ws.xsigo.com> <1213270788.14047.451.camel@hrosenstock-ws.xsigo.com> <1213278134.14047.498.camel@hrosenstock-ws.xsigo.com> <1214509372.13056.645.camel@hrosenstock-ws.xsigo.com> Message-ID: <1215533144.18371.109.camel@hrosenstock-ws.xsigo.com> Hi Olga, On Tue, 2008-07-08 at 18:27 +0300, Olga Shern (Voltaire) wrote: > Hi, > > I have tested the same scenario with OpenSM. > What I have expected to see is trap 259 because of switch enforcement > that is done by OpenSM. I don't see this trap, therefore I assume that > ANAFA Do you mean Anafa II ? > doesn't generate this trap (this is not mandatory only > optional). Right, this trap is optional and I don't know whether or not it is supported. Perhaps someone from Mellanox can comment on this to be sure. > I only see trap 257 that is generated by the node (with partial pkey) > that sends the multicast traffic. Only when there is no switch enforcement ? > The conclusion is that if there is switch enforcement the issue is not > severe and we can live with it :) Would that be the case if those optional traps were supported ? Thanks. -- Hal > Olga > > On 6/26/08, Hal Rosenstock wrote: > > On Thu, 2008-06-12 at 16:44 +0300, Olga Shern (Voltaire) wrote: > > > Would you try this with OpenSM (and validate your theory about > > > getting > > > switch bad PKey traps v. end port bad PKey traps) or does VSM > > > have such > > > a mode (ingress/egress partition filtering) ? > > > > > > -- Hal > > > > > > Yes, I will test it with OpenSM > > > > Any update on this ? Thanks. > > > > -- Hal > > > > From Jeffrey.C.Becker at nasa.gov Tue Jul 8 10:07:33 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Tue, 08 Jul 2008 10:07:33 -0700 Subject: [ofa-general] rping debugging Message-ID: <48739ED5.2080106@nasa.gov> Hi all. I have two hosts connected by an infiniband cable. One is an Athlon 64 node with an mthca card, and the other is an Opteron server with a ConnectX card. They are both running kernel.org 2.6.25.9 including the associated IB modules. I have the fabric up and simple ib commands as well as IP over IB run correctly. In the interest of testing NFS over RDMA, I'm trying to verify my rdmacm setup using rping (Thanks to Tom Tucker and James Lentini for pointing me in this direction). I installed and built librdmacm-1.0.7 on both nodes. Regardless of which server/client assignment I make, I get: $ rping -d -s server created cm_id 0x6072d0 rdma_bind_addr successful rdma_listen (waits here....) $ rping -d -c 10.1.2.4 client created cm_id 0x506130 cma_event type RDMA_CM_EVENT_ADDR_RESOLVED cma_id 0x506130 (parent) cma_event type RDMA_CM_EVENT_ROUTE_RESOLVED cma_id 0x506130 (parent) rdma_resolve_addr - rdma_resolve_route successful created pd 0x506480 created channel 0x5064a0 created cq 0x5088f0 created qp 0x508a20 rping_setup_buffers called on cb 0x505010 allocated & registered buffers... cq_thread started. cq completion failed status 5 wait for CONNECTED state 10 connect error -1 rping_free_buffers called on cb 0x505010 cma_event type RDMA_CM_EVENT_REJECTED cma_id 0x506130 (parent) cma event RDMA_CM_EVENT_REJECTED, error 8 I'm going to run client and server through gdb to pin this down further, but I was wondering if anyone had any suggestions? Thanks. -jeff From rdreier at cisco.com Tue Jul 8 10:14:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 10:14:11 -0700 Subject: [ofa-general] rping debugging In-Reply-To: <48739ED5.2080106@nasa.gov> (Jeff Becker's message of "Tue, 08 Jul 2008 10:07:33 -0700") References: <48739ED5.2080106@nasa.gov> Message-ID: > $ rping -d -c 10.1.2.4 I believe you want to run rping -d -c -a 10.1.2.4 ie the address is part of the "-a" command line option. It's a little unfortunate that rping doesn't warn you if it doesn't see a server address when running as the client. From olga.shern at gmail.com Tue Jul 8 10:21:27 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Tue, 8 Jul 2008 20:21:27 +0300 Subject: [ofa-general] Multicast traffic generates Bad P_Key trap in SM when working in partial member setup In-Reply-To: <1215533144.18371.109.camel@hrosenstock-ws.xsigo.com> References: <39C75744D164D948A170E9792AF8E7CA013E6E2B@exil.voltaire.com> <1213266626.14047.419.camel@hrosenstock-ws.xsigo.com> <1213270788.14047.451.camel@hrosenstock-ws.xsigo.com> <1213278134.14047.498.camel@hrosenstock-ws.xsigo.com> <1214509372.13056.645.camel@hrosenstock-ws.xsigo.com> <1215533144.18371.109.camel@hrosenstock-ws.xsigo.com> Message-ID: On 7/8/08, Hal Rosenstock wrote: > Hi Olga, > > On Tue, 2008-07-08 at 18:27 +0300, Olga Shern (Voltaire) wrote: > > Hi, > > > > I have tested the same scenario with OpenSM. > > What I have expected to see is trap 259 because of switch enforcement > > that is done by OpenSM. I don't see this trap, therefore I assume that > > ANAFA > > Do you mean Anafa II ? > yes > > doesn't generate this trap (this is not mandatory only > > optional). > > Right, this trap is optional and I don't know whether or not it is > supported. Perhaps someone from Mellanox can comment on this to be sure. > > > I only see trap 257 that is generated by the node (with partial pkey) > > that sends the multicast traffic. > > Only when there is no switch enforcement ? > no, when there is switch enforcement, I see this trap generated by the node that sends multicast packets > > The conclusion is that if there is switch enforcement the issue is not > > severe and we can live with it :) > > Would that be the case if those optional traps were supported ? > I think no > Thanks. > -- Hal > > > Olga > > > > On 6/26/08, Hal Rosenstock wrote: > > > On Thu, 2008-06-12 at 16:44 +0300, Olga Shern (Voltaire) wrote: > > > > Would you try this with OpenSM (and validate your theory about > > > > getting > > > > switch bad PKey traps v. end port bad PKey traps) or does VSM > > > > have such > > > > a mode (ingress/egress partition filtering) ? > > > > > > > > -- Hal > > > > > > > > Yes, I will test it with OpenSM > > > > > > Any update on this ? Thanks. > > > > > > -- Hal > > > > > > > > From Jeffrey.C.Becker at nasa.gov Tue Jul 8 10:28:32 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Tue, 08 Jul 2008 10:28:32 -0700 Subject: [ofa-general] rping debugging In-Reply-To: References: <48739ED5.2080106@nasa.gov> Message-ID: <4873A3C0.2010600@nasa.gov> Hi Roland. Roland Dreier wrote: > > $ rping -d -c 10.1.2.4 > > I believe you want to run > > rping -d -c -a 10.1.2.4 > > ie the address is part of the "-a" command line option. > Thanks. Yes you are right. That works fine. My bad - I know, RTFM (actually I scanned it but must have missed the -a somehow) :-[ . Sorry for spamming the list. -jeff > It's a little unfortunate that rping doesn't warn you if it doesn't see > a server address when running as the client. > From rdreier at cisco.com Tue Jul 8 10:47:54 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 10:47:54 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <20080701201708.20859.49697.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 01 Jul 2008 15:17:08 -0500") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> Message-ID: thanks, applied. From rdreier at cisco.com Tue Jul 8 11:00:45 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:00:45 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: (Roland Dreier's message of "Tue, 08 Jul 2008 10:47:54 -0700") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> Message-ID: so I applied this, but now I'm starting to wonder about how ULPs would use this. As I understand things, this is really working around the fact that cxgb3's get_dma_mr() implementation is limited to 4G. But how can a ULP know if a given device has the 4G limitation? For every other device (modulo iw_nes's 1 TB limitation) get_dma_mr works fine for local DMA. Thoughts? - R. From swise at opengridcomputing.com Tue Jul 8 11:07:04 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 13:07:04 -0500 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807081400.17025.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807062138.29103.olaf.kirch@oracle.com> <48733D5C.7060000@voltaire.com> <200807081400.17025.olaf.kirch@oracle.com> Message-ID: <4873ACC8.7070804@opengridcomputing.com> Olaf Kirch wrote: > On Tuesday 08 July 2008 12:11:40 Or Gerlitz wrote: > >> Olaf Kirch wrote: >> >>> Okay, there's a third way to fix this, which may even work with today's Oracle IPC code. The hack is to not submit the fastreg WR until we post a SEND that transmits the rkey to a peer. Then >>> >>> - we have a QP to post the WRs to >>> - we post the fastreg and the SEND to the same QP, >>> ensuring that the peer doesn't initiate the RDMA >>> before we've set up the memory registration. >>> >> Yes, as Steve noted, using the same QP for the fastreg and send that >> carries the rkey/stag is a must since the send should be fenced on the >> fastreg completion to make sure the remote side uses a valid key. >> > > One theoretical problem is what happens if the application wants to > register a memory range and send the rkey to two peers at the same time? > We would have to stall the second send explicitly as long as we > haven't seen the completion for the first send. > Or signal the fast_reg wr and only complete the application's register when the fastreg wr completes... From rdreier at cisco.com Tue Jul 8 11:07:36 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:07:36 -0700 Subject: [ofa-general] [PATCH for-2.6.27 2/2] RDMA/cxgb3: Fixes for zero-stag. In-Reply-To: <20080701201710.20859.39200.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 01 Jul 2008 15:17:10 -0500") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201710.20859.39200.stgit@dell3.ogc.int> Message-ID: thanks, applied From swise at opengridcomputing.com Tue Jul 8 11:12:24 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 13:12:24 -0500 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> Message-ID: <4873AE08.2070205@opengridcomputing.com> Roland Dreier wrote: > so I applied this, but now I'm starting to wonder about how ULPs would > use this. As I understand things, this is really working around the > fact that cxgb3's get_dma_mr() implementation is limited to 4G. But how > can a ULP know if a given device has the 4G limitation? For every other > device (modulo iw_nes's 1 TB limitation) get_dma_mr works fine for local > DMA. Thoughts? > > - R. > RDS and NFSRDMA will use the local_dma_lkey if the device supports it. If not, it will use a dma_mr with local access rights. So using the local_dma_lkey is just an optimization to avoid allocating a local-access-only dma_mr. Steve. From rdreier at cisco.com Tue Jul 8 11:16:23 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:16:23 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <4873AE08.2070205@opengridcomputing.com> (Steve Wise's message of "Tue, 08 Jul 2008 13:12:24 -0500") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> Message-ID: > RDS and NFSRDMA will use the local_dma_lkey if the device supports it. > If not, it will use a dma_mr with local access rights. So using the > local_dma_lkey is just an optimization to avoid allocating a > local-access-only dma_mr. I wonder if it's worth it... the alternative would be for cxgb3 etc. to just return STag 0 from get_dma_mr if only local access rights are requested. But of course as we discussed later STag 0 doesn't do PD protection, if anyone cared. We can always change it later anyway. - R. From swise at opengridcomputing.com Tue Jul 8 11:30:57 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 13:30:57 -0500 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> Message-ID: <4873B261.4060903@opengridcomputing.com> Roland Dreier wrote: > > RDS and NFSRDMA will use the local_dma_lkey if the device supports it. > > If not, it will use a dma_mr with local access rights. So using the > > local_dma_lkey is just an optimization to avoid allocating a > > local-access-only dma_mr. > > I wonder if it's worth it... the alternative would be for cxgb3 etc. to > just return STag 0 from get_dma_mr if only local access rights are > requested. But of course as we discussed later STag 0 doesn't do PD > protection, if anyone cared. > > I assumed that since the PD protection couldn't be enforced, that cxgb3 shouldn't take this approach. But yea, if we ignore the pd issue, cxgb3 could hide this in a dma-mr, for local access only. Stevo. From rdreier at cisco.com Tue Jul 8 11:39:01 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:39:01 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <4873B261.4060903@opengridcomputing.com> (Steve Wise's message of "Tue, 08 Jul 2008 13:30:57 -0500") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> <4873B261.4060903@opengridcomputing.com> Message-ID: > I assumed that since the PD protection couldn't be enforced, that > cxgb3 shouldn't take this approach. But yea, if we ignore the pd > issue, cxgb3 could hide this in a dma-mr, for local access only. I guess this is probably the right idea. It's a little unfortunate that ULPs have to try the local_lkey and then fall back to get_dma_mr() but oh well. From swise at opengridcomputing.com Tue Jul 8 11:40:54 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 13:40:54 -0500 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> Message-ID: <4873B4B6.4040604@opengridcomputing.com> Roland Dreier wrote: > > RDS and NFSRDMA will use the local_dma_lkey if the device supports it. > > If not, it will use a dma_mr with local access rights. So using the > > local_dma_lkey is just an optimization to avoid allocating a > > local-access-only dma_mr. > > I wonder if it's worth it... the alternative would be for cxgb3 etc. to > just return STag 0 from get_dma_mr if only local access rights are > requested. But of course as we discussed later STag 0 doesn't do PD > protection, if anyone cared. > > We can always change it later anyway. > > - R. > I could still change cxgb3 to do this if you want. We can have both... From swise at opengridcomputing.com Tue Jul 8 11:42:22 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 13:42:22 -0500 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> <4873B261.4060903@opengridcomputing.com> Message-ID: <4873B50E.6030908@opengridcomputing.com> Roland Dreier wrote: > > I assumed that since the PD protection couldn't be enforced, that > > cxgb3 shouldn't take this approach. But yea, if we ignore the pd > > issue, cxgb3 could hide this in a dma-mr, for local access only. > > I guess this is probably the right idea. It's a little unfortunate that > ULPs have to try the local_lkey and then fall back to get_dma_mr() but > oh well. > Do all the current IB hca's support the local_dma_lkey? I mean, do we really need a capability bit? From rdreier at cisco.com Tue Jul 8 11:43:48 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:43:48 -0700 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking needed for serialization of callbacks In-Reply-To: (Or Gerlitz's message of "Thu, 19 Jun 2008 17:07:01 +0300 (IDT)") References: Message-ID: > @@ -2570,9 +2557,12 @@ static int cma_ib_mc_handler(int status, > int ret; > > id_priv = mc->id_priv; > - if (cma_disable_remove(id_priv, CMA_ADDR_BOUND) && > - cma_disable_remove(id_priv, CMA_ADDR_RESOLVED)) > + mutex_lock(&id_priv->handler_mutex); > + if ((id_priv->state != CMA_ADDR_BOUND) && > + (id_priv->state != CMA_ADDR_RESOLVED)) { > + mutex_unlock(&id_priv->handler_mutex); > return 0; > + } why don't you just do if (cma_disable_callback(id_priv, CMA_ADDR_BOUND) && cma_disable_callback(id_priv, CMA_ADDR_RESOLVED)) instead of open-coding here? (&& is specified to have short-circuit semantics, so if the first call takes the mutex, the second call won't be made) It seems your change is a subtle change in semantics and breaks the encapsulation of this callback disabling here. - R. From rdreier at cisco.com Tue Jul 8 11:45:03 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:45:03 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: <4873B50E.6030908@opengridcomputing.com> (Steve Wise's message of "Tue, 08 Jul 2008 13:42:22 -0500") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> <4873B261.4060903@opengridcomputing.com> <4873B50E.6030908@opengridcomputing.com> Message-ID: > Do all the current IB hca's support the local_dma_lkey? I mean, do we > really need a capability bit? Not all Mellanox HCAs do -- I think Arbel/Sinai would need FW updates, and I don't think Tavor could ever do it. And ipath hasn't implemented it, and I don't know about ehca. - R. From sean.hefty at intel.com Tue Jul 8 11:55:27 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 8 Jul 2008 11:55:27 -0700 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking neededfor serialization of callbacks In-Reply-To: References: Message-ID: <000c01c8e12c$2ff11b00$a8c8180a@amr.corp.intel.com> >why don't you just do > > if (cma_disable_callback(id_priv, CMA_ADDR_BOUND) && > cma_disable_callback(id_priv, CMA_ADDR_RESOLVED)) > >instead of open-coding here? (&& is specified to have short-circuit >semantics, so if the first call takes the mutex, the second call won't >be made) > >It seems your change is a subtle change in semantics and breaks the >encapsulation of this callback disabling here. Personally, I'm fine either way. This method can acquire/release the mutex twice, though that's not a big deal. If we want better encapsulation, we could also add a cma_enable_callback wrapper around the mutex_unlock. - Sean From rdreier at cisco.com Tue Jul 8 11:58:28 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 11:58:28 -0700 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking neededfor serialization of callbacks In-Reply-To: <000c01c8e12c$2ff11b00$a8c8180a@amr.corp.intel.com> (Sean Hefty's message of "Tue, 8 Jul 2008 11:55:27 -0700") References: <000c01c8e12c$2ff11b00$a8c8180a@amr.corp.intel.com> Message-ID: > Personally, I'm fine either way. This method can acquire/release the mutex > twice, though that's not a big deal. If we want better encapsulation, we could > also add a cma_enable_callback wrapper around the mutex_unlock. I was thinking about that enable wrapper too... might make everything easier to read. (I think the double mutex in some cases is totally negligible, and the clearer source more than makes up for it) From swise at opengridcomputing.com Tue Jul 8 12:17:57 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 08 Jul 2008 14:17:57 -0500 Subject: [ofa-general] [PATCH for-2.6.27] RDMA/cxgb3: Fix regression from commit f4e91eb4a81559da87a3843758a641b5cc590b65. Message-ID: <20080708191757.12304.42287.stgit@dell3.ogc.int> From: Steve Wise Roland, Dunno if this is the correct fix or not based on the netdev discussion, but this works. :) Steve --- RDMA/cxgb3: Fix regression from commit f4e91eb4a81559da87a3843758a641b5cc590b65. The change to iwch_provider.c in commit f4e91eb4a81559da87a3843758a641b5cc590b65 undid the fix done in commit 7f049f2f4259e4a5c213720df1ce0c2ca095641a. It removed the calls to rtnl_lock() that serialized the iw_cxgb3 ethtool ops calls into the cxgb3 driver. Also, there were some more recent additions that call ethtool ops without the rtnl_lock held. Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/iwch_provider.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index d54aace..c261c3c 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1102,7 +1102,9 @@ static u64 fw_vers_string_to_u64(struct iwch_dev *iwch_dev) char *cp, *next; unsigned fw_maj, fw_min, fw_mic; + rtnl_lock(); lldev->ethtool_ops->get_drvinfo(lldev, &info); + rtnl_unlock(); next = info.fw_version + 1; cp = strsep(&next, "."); sscanf(cp, "%i", &fw_maj); @@ -1190,7 +1192,9 @@ static int fw_supports_fastreg(struct iwch_dev *iwch_dev) char *cp, *next; unsigned fw_maj, fw_min; + rtnl_lock(); lldev->ethtool_ops->get_drvinfo(lldev, &info); + rtnl_unlock(); next = info.fw_version+1; cp = strsep(&next, "."); sscanf(cp, "%i", &fw_maj); @@ -1208,7 +1212,9 @@ static ssize_t show_fw_ver(struct device *dev, struct device_attribute *attr, ch struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev; PDBG("%s dev 0x%p\n", __func__, dev); + rtnl_lock(); lldev->ethtool_ops->get_drvinfo(lldev, &info); + rtnl_unlock(); return sprintf(buf, "%s\n", info.fw_version); } @@ -1221,7 +1227,9 @@ static ssize_t show_hca(struct device *dev, struct device_attribute *attr, struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev; PDBG("%s dev 0x%p\n", __func__, dev); + rtnl_lock(); lldev->ethtool_ops->get_drvinfo(lldev, &info); + rtnl_unlock(); return sprintf(buf, "%s\n", info.driver); } From tziporet at dev.mellanox.co.il Tue Jul 8 12:33:59 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 08 Jul 2008 22:33:59 +0300 Subject: [ofa-general] Re: [PATCH for-2.6.27 1/2] RDMA/Core: Add local dma lkey support. In-Reply-To: References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201708.20859.49697.stgit@dell3.ogc.int> <4873AE08.2070205@opengridcomputing.com> <4873B261.4060903@opengridcomputing.com> <4873B50E.6030908@opengridcomputing.com> Message-ID: <4873C127.9080105@mellanox.co.il> Roland Dreier wrote: > > Do all the current IB hca's support the local_dma_lkey? I mean, do we > > really need a capability bit? > > Not all Mellanox HCAs do -- I think Arbel/Sinai would need FW updates, > and I don't think Tavor could ever do it. We do not plan to add this to Arbel/Sinai. Only ConnectX will support it Tziporet From or.gerlitz at gmail.com Tue Jul 8 12:59:19 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 8 Jul 2008 22:59:19 +0300 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking neededfor serialization of callbacks In-Reply-To: References: <000c01c8e12c$2ff11b00$a8c8180a@amr.corp.intel.com> Message-ID: <15ddcffd0807081259u47ed2e38i7388f9d38fd91f88@mail.gmail.com> On 7/8/08, Roland Dreier wrote: > > > Personally, I'm fine either way. This method can acquire/release the > mutex > > twice, though that's not a big deal. If we want better encapsulation, > we could > > also add a cma_enable_callback wrapper around the mutex_unlock. > > I was thinking about that enable wrapper too... might make everything > easier to read. (I think the double mutex in some cases is totally > negligible, and the clearer source more than makes up for it) > OK, I just want to make sure I got exactly what you have in mind: this patch should be fixed to have a if (cma_disable_callback(id_priv, CMA_ADDR_BOUND) && cma_disable_callback(id_priv, CMA_ADDR_RESOLVED)) calls in cma_ib_mc_handler and have a cma_enable_callback wrapper around the mutex_unlock, correct? I will be able to do that on Thursday. the reason for the open coding in cma_ib_mc_handler was an attempt to stay away from the compiler and avoid potential double locking of a mutex. The reason for not having a wrapper was that for my taste it was a bit of over doing, but I am fine with having it if this is your preference as maintainers. Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Jul 8 13:02:38 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 13:02:38 -0700 Subject: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking neededfor serialization of callbacks In-Reply-To: <15ddcffd0807081259u47ed2e38i7388f9d38fd91f88@mail.gmail.com> (Or Gerlitz's message of "Tue, 8 Jul 2008 22:59:19 +0300") References: <000c01c8e12c$2ff11b00$a8c8180a@amr.corp.intel.com> <15ddcffd0807081259u47ed2e38i7388f9d38fd91f88@mail.gmail.com> Message-ID: > OK, I just want to make sure I got exactly what you have in mind: > > this patch should be fixed to have a > > if (cma_disable_callback(id_priv, CMA_ADDR_BOUND) && > cma_disable_callback(id_priv, CMA_ADDR_RESOLVED)) > > calls in cma_ib_mc_handler and have a cma_enable_callback wrapper around > the mutex_unlock, correct? Actually I fixed this up and just applied patches 1 and 2 already. After I read the code more closely I decided that the cma_enable_callback() wrapper didn't make sense, because there are already so many places that have to use the handler_mutex by hand. So this is what I have committed: commit 9cc106fc4da853dacc4de69a607670b90892921f Author: Or Gerlitz Date: Tue Jul 8 13:00:30 2008 -0700 RDMA/cma: Simplify locking needed for serialization of callbacks The RDMA CM has some logic in place to make sure that callbacks on a given CM ID are delivered to the consumer in a serialized manner. Specifically it has code to protect against a device removal racing with a running callback function. This patch simplifies this logic by using a mutex per ID instead of a wait queue and atomic variable. This means that cma_disable_remove() now is more properly named to cma_disable_callback(), and cma_enable_remove() can now be removed because it just would become a trivial wrapper around mutex_unlock(). Signed-off-by: Or Gerlitz Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 5fb506a..0e7239e 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -130,8 +130,7 @@ struct rdma_id_private { struct completion comp; atomic_t refcount; - wait_queue_head_t wait_remove; - atomic_t dev_remove; + struct mutex handler_mutex; int backlog; int timeout_ms; @@ -355,26 +354,15 @@ static void cma_deref_id(struct rdma_id_private *id_priv) complete(&id_priv->comp); } -static int cma_disable_remove(struct rdma_id_private *id_priv, +static int cma_disable_callback(struct rdma_id_private *id_priv, enum cma_state state) { - unsigned long flags; - int ret; - - spin_lock_irqsave(&id_priv->lock, flags); - if (id_priv->state == state) { - atomic_inc(&id_priv->dev_remove); - ret = 0; - } else - ret = -EINVAL; - spin_unlock_irqrestore(&id_priv->lock, flags); - return ret; -} - -static void cma_enable_remove(struct rdma_id_private *id_priv) -{ - if (atomic_dec_and_test(&id_priv->dev_remove)) - wake_up(&id_priv->wait_remove); + mutex_lock(&id_priv->handler_mutex); + if (id_priv->state != state) { + mutex_unlock(&id_priv->handler_mutex); + return -EINVAL; + } + return 0; } static int cma_has_cm_dev(struct rdma_id_private *id_priv) @@ -399,8 +387,7 @@ struct rdma_cm_id *rdma_create_id(rdma_cm_event_handler event_handler, mutex_init(&id_priv->qp_mutex); init_completion(&id_priv->comp); atomic_set(&id_priv->refcount, 1); - init_waitqueue_head(&id_priv->wait_remove); - atomic_set(&id_priv->dev_remove, 0); + mutex_init(&id_priv->handler_mutex); INIT_LIST_HEAD(&id_priv->listen_list); INIT_LIST_HEAD(&id_priv->mc_list); get_random_bytes(&id_priv->seq_num, sizeof id_priv->seq_num); @@ -927,7 +914,7 @@ static int cma_ib_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) struct rdma_cm_event event; int ret = 0; - if (cma_disable_remove(id_priv, CMA_CONNECT)) + if (cma_disable_callback(id_priv, CMA_CONNECT)) return 0; memset(&event, 0, sizeof event); @@ -984,12 +971,12 @@ static int cma_ib_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) /* Destroy the CM ID by returning a non-zero value. */ id_priv->cm_id.ib = NULL; cma_exch(id_priv, CMA_DESTROYING); - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); rdma_destroy_id(&id_priv->id); return ret; } out: - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); return ret; } @@ -1101,7 +1088,7 @@ static int cma_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) int offset, ret; listen_id = cm_id->context; - if (cma_disable_remove(listen_id, CMA_LISTEN)) + if (cma_disable_callback(listen_id, CMA_LISTEN)) return -ECONNABORTED; memset(&event, 0, sizeof event); @@ -1122,7 +1109,7 @@ static int cma_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) goto out; } - atomic_inc(&conn_id->dev_remove); + mutex_lock(&conn_id->handler_mutex); mutex_lock(&lock); ret = cma_acquire_dev(conn_id); mutex_unlock(&lock); @@ -1144,7 +1131,7 @@ static int cma_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) !cma_is_ud_ps(conn_id->id.ps)) ib_send_cm_mra(cm_id, CMA_CM_MRA_SETTING, NULL, 0); mutex_unlock(&lock); - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); goto out; } @@ -1153,11 +1140,11 @@ static int cma_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) release_conn_id: cma_exch(conn_id, CMA_DESTROYING); - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(&conn_id->id); out: - cma_enable_remove(listen_id); + mutex_unlock(&listen_id->handler_mutex); return ret; } @@ -1223,7 +1210,7 @@ static int cma_iw_handler(struct iw_cm_id *iw_id, struct iw_cm_event *iw_event) struct sockaddr_in *sin; int ret = 0; - if (cma_disable_remove(id_priv, CMA_CONNECT)) + if (cma_disable_callback(id_priv, CMA_CONNECT)) return 0; memset(&event, 0, sizeof event); @@ -1267,12 +1254,12 @@ static int cma_iw_handler(struct iw_cm_id *iw_id, struct iw_cm_event *iw_event) /* Destroy the CM ID by returning a non-zero value. */ id_priv->cm_id.iw = NULL; cma_exch(id_priv, CMA_DESTROYING); - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); rdma_destroy_id(&id_priv->id); return ret; } - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); return ret; } @@ -1288,7 +1275,7 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, struct ib_device_attr attr; listen_id = cm_id->context; - if (cma_disable_remove(listen_id, CMA_LISTEN)) + if (cma_disable_callback(listen_id, CMA_LISTEN)) return -ECONNABORTED; /* Create a new RDMA id for the new IW CM ID */ @@ -1300,19 +1287,19 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, goto out; } conn_id = container_of(new_cm_id, struct rdma_id_private, id); - atomic_inc(&conn_id->dev_remove); + mutex_lock(&conn_id->handler_mutex); conn_id->state = CMA_CONNECT; dev = ip_dev_find(&init_net, iw_event->local_addr.sin_addr.s_addr); if (!dev) { ret = -EADDRNOTAVAIL; - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(new_cm_id); goto out; } ret = rdma_copy_addr(&conn_id->id.route.addr.dev_addr, dev, NULL); if (ret) { - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(new_cm_id); goto out; } @@ -1321,7 +1308,7 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, ret = cma_acquire_dev(conn_id); mutex_unlock(&lock); if (ret) { - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(new_cm_id); goto out; } @@ -1337,7 +1324,7 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, ret = ib_query_device(conn_id->id.device, &attr); if (ret) { - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(new_cm_id); goto out; } @@ -1353,14 +1340,14 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, /* User wants to destroy the CM ID */ conn_id->cm_id.iw = NULL; cma_exch(conn_id, CMA_DESTROYING); - cma_enable_remove(conn_id); + mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(&conn_id->id); } out: if (dev) dev_put(dev); - cma_enable_remove(listen_id); + mutex_unlock(&listen_id->handler_mutex); return ret; } @@ -1592,7 +1579,7 @@ static void cma_work_handler(struct work_struct *_work) struct rdma_id_private *id_priv = work->id; int destroy = 0; - atomic_inc(&id_priv->dev_remove); + mutex_lock(&id_priv->handler_mutex); if (!cma_comp_exch(id_priv, work->old_state, work->new_state)) goto out; @@ -1601,7 +1588,7 @@ static void cma_work_handler(struct work_struct *_work) destroy = 1; } out: - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); cma_deref_id(id_priv); if (destroy) rdma_destroy_id(&id_priv->id); @@ -1764,7 +1751,7 @@ static void addr_handler(int status, struct sockaddr *src_addr, struct rdma_cm_event event; memset(&event, 0, sizeof event); - atomic_inc(&id_priv->dev_remove); + mutex_lock(&id_priv->handler_mutex); /* * Grab mutex to block rdma_destroy_id() from removing the device while @@ -1793,13 +1780,13 @@ static void addr_handler(int status, struct sockaddr *src_addr, if (id_priv->id.event_handler(&id_priv->id, &event)) { cma_exch(id_priv, CMA_DESTROYING); - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); cma_deref_id(id_priv); rdma_destroy_id(&id_priv->id); return; } out: - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); cma_deref_id(id_priv); } @@ -2126,7 +2113,7 @@ static int cma_sidr_rep_handler(struct ib_cm_id *cm_id, struct ib_cm_sidr_rep_event_param *rep = &ib_event->param.sidr_rep_rcvd; int ret = 0; - if (cma_disable_remove(id_priv, CMA_CONNECT)) + if (cma_disable_callback(id_priv, CMA_CONNECT)) return 0; memset(&event, 0, sizeof event); @@ -2167,12 +2154,12 @@ static int cma_sidr_rep_handler(struct ib_cm_id *cm_id, /* Destroy the CM ID by returning a non-zero value. */ id_priv->cm_id.ib = NULL; cma_exch(id_priv, CMA_DESTROYING); - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); rdma_destroy_id(&id_priv->id); return ret; } out: - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); return ret; } @@ -2570,8 +2557,8 @@ static int cma_ib_mc_handler(int status, struct ib_sa_multicast *multicast) int ret; id_priv = mc->id_priv; - if (cma_disable_remove(id_priv, CMA_ADDR_BOUND) && - cma_disable_remove(id_priv, CMA_ADDR_RESOLVED)) + if (cma_disable_callback(id_priv, CMA_ADDR_BOUND) && + cma_disable_callback(id_priv, CMA_ADDR_RESOLVED)) return 0; mutex_lock(&id_priv->qp_mutex); @@ -2596,12 +2583,12 @@ static int cma_ib_mc_handler(int status, struct ib_sa_multicast *multicast) ret = id_priv->id.event_handler(&id_priv->id, &event); if (ret) { cma_exch(id_priv, CMA_DESTROYING); - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); rdma_destroy_id(&id_priv->id); return 0; } - cma_enable_remove(id_priv); + mutex_unlock(&id_priv->handler_mutex); return 0; } @@ -2760,6 +2747,7 @@ static int cma_remove_id_dev(struct rdma_id_private *id_priv) { struct rdma_cm_event event; enum cma_state state; + int ret = 0; /* Record that we want to remove the device */ state = cma_exch(id_priv, CMA_DEVICE_REMOVAL); @@ -2767,15 +2755,18 @@ static int cma_remove_id_dev(struct rdma_id_private *id_priv) return 0; cma_cancel_operation(id_priv, state); - wait_event(id_priv->wait_remove, !atomic_read(&id_priv->dev_remove)); + mutex_lock(&id_priv->handler_mutex); /* Check for destruction from another callback. */ if (!cma_comp(id_priv, CMA_DEVICE_REMOVAL)) - return 0; + goto out; memset(&event, 0, sizeof event); event.event = RDMA_CM_EVENT_DEVICE_REMOVAL; - return id_priv->id.event_handler(&id_priv->id, &event); + ret = id_priv->id.event_handler(&id_priv->id, &event); +out: + mutex_unlock(&id_priv->handler_mutex); + return ret; } static void cma_process_remove(struct cma_device *cma_dev) From rdreier at cisco.com Tue Jul 8 13:08:31 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 13:08:31 -0700 Subject: [ofa-general] Re: [PATCH for-2.6.27] RDMA/cxgb3: Fix regression from commit f4e91eb4a81559da87a3843758a641b5cc590b65. In-Reply-To: <20080708191757.12304.42287.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 08 Jul 2008 14:17:57 -0500") References: <20080708191757.12304.42287.stgit@dell3.ogc.int> Message-ID: > Also, there were some more recent additions that call ethtool ops without > the rtnl_lock held. I just rolled rtnl_lock/unlock calls into the new get_drvinfo calls in patches in the for-2.6.27 branch that aren't in Linus's tree yet. - R. From jon at opengridcomputing.com Tue Jul 8 13:27:55 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Tue, 8 Jul 2008 15:27:55 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807071440.46502.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807071440.46502.olaf.kirch@oracle.com> Message-ID: <20080708202755.GB31809@opengridcomputing.com> Hey Olaf, With the patch below on top of your previous patch, I am able to get it working over iWARP (though RDMA is still breaking, but in a new and different way). The patch modifies rds_ib_laddr_check to use rdma_bind_addr to determine if it is a IB/iWARP device by creating a temporary rdma_cm_id and binding to the address passed in. I'm sure this can be optimized, but I'll leave that as a todo for later. This patch reduces the default send_wr and recv_wr to sizes that Chelsio can handle, as the sizes before were too large. Optimally, this should be determined with a query of the device. I have added that to my todo list as well The lguid and fguid cannot be used for iWARP as those structs do not exists and give off a nasty illegal pointer deference. I removed the references to those, as the appear to only be for debug messages. Lastly, this has the port space differentiation (which is the only way TCP and iWARP can happily co-exist). Let me know what you think. Thanks, Jon Signed-off-by: Jon Mason diff --git a/net/rds/ib.c b/net/rds/ib.c index cd2dc7c..067fab0 100644 --- a/net/rds/ib.c +++ b/net/rds/ib.c @@ -158,17 +158,29 @@ void rds_ib_remove_one(struct ib_device *device) */ static int rds_ib_laddr_check(__be32 addr) { - struct net_device *dev; - int ret; + int ret = -EADDRNOTAVAIL; + struct rdma_cm_id *cm_id; + struct sockaddr_in sin; + + cm_id = rdma_create_id(NULL, NULL, RDMA_PS_TCP); + if (cm_id == NULL) { + printk("rdma_create_id failed\n"); + return -EADDRNOTAVAIL; + } - dev = ip_dev_find(addr); - if (dev && dev->type == ARPHRD_INFINIBAND) { - dev_put(dev); - ret = 0; - } else + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_addr.s_addr = addr; + + /* rdma_bind_addr will fail for non-IB/iWARP devices */ + ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin); + if (ret) ret = -EADDRNOTAVAIL; - rdsdebug("addr %u.%u.%u.%u ret %d\n", NIPQUAD(addr), ret); + rdsdebug("addr %u.%u.%u.%u ret %d node type %d\n", + NIPQUAD(addr), ret, cm_id->device?cm_id->device->node_type:-1); + + rdma_destroy_id(cm_id); return ret; } diff --git a/net/rds/ib.h b/net/rds/ib.h index 947977c..2924fda 100644 --- a/net/rds/ib.h +++ b/net/rds/ib.h @@ -7,14 +7,16 @@ #define RDS_IB_RESOLVE_TIMEOUT_MS 5000 -#define RDS_FMR_SIZE 256 +/* FIXME - use ib_query_device to determine proper value */ +#define RDS_FMR_SIZE 20 #define RDS_FMR_POOL_SIZE 2048 -#define RDS_IB_MAX_SGE 8 +#define RDS_IB_MAX_SGE 8 #define RDS_IB_RECV_SGE 2 -#define RDS_IB_DEFAULT_RECV_WR 1024 -#define RDS_IB_DEFAULT_SEND_WR 256 +/* FIXME - call ib_query_device to determine sane values for this based on the HW */ +#define RDS_IB_DEFAULT_RECV_WR 512 +#define RDS_IB_DEFAULT_SEND_WR 128 #define RDS_IB_SUPPORTED_PROTOCOLS 0x00000003 /* minor versions supported */ diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index a49e394..fd56481 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -296,7 +296,7 @@ static int rds_ib_setup_qp(struct rds_connection *conn) */ ret = rdma_create_qp(ic->i_cm_id, ic->i_pd, &attr); if (ret) { - rdsdebug("ib_req_notify_cq failed: %d\n", ret); + rdsdebug("rdma_create_qp failed: %d\n", ret); goto out; } @@ -388,8 +388,6 @@ static u32 rds_ib_protocol_compatible(const struct rds_ib_connect_private *dp) static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, struct rdma_cm_event *event) { - __be64 lguid = cm_id->route.path_rec->sgid.global.interface_id; - __be64 fguid = cm_id->route.path_rec->dgid.global.interface_id; const struct rds_ib_connect_private *dp = event->param.conn.private_data; struct rds_ib_connect_private dp_rep; struct rds_connection *conn = NULL; @@ -404,11 +402,9 @@ static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, if (!version) goto out; - rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u lguid 0x%llx fguid " - "0x%llx\n", NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), - RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version), - (unsigned long long)be64_to_cpu(lguid), - (unsigned long long)be64_to_cpu(fguid)); + rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u\n", + NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), + RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version)); conn = rds_conn_create(dp->dp_daddr, dp->dp_saddr, &rds_ib_transport, GFP_KERNEL); @@ -628,7 +624,7 @@ int rds_ib_conn_connect(struct rds_connection *conn) dest.sin_family = AF_INET; dest.sin_addr.s_addr = (__force u32)conn->c_faddr; - dest.sin_port = (__force u16)htons(RDS_PORT); + dest.sin_port = (__force u16)htons(RDS_IB_PORT); ret = rdma_resolve_addr(ic->i_cm_id, (struct sockaddr *)&src, (struct sockaddr *)&dest, @@ -813,7 +809,7 @@ int __init rds_ib_listen_init(void) sin.sin_family = PF_INET, sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); - sin.sin_port = (__force u16)htons(RDS_PORT); + sin.sin_port = (__force u16)htons(RDS_IB_PORT); /* * XXX I bet this binds the cm_id to a device. If we want to support @@ -833,7 +829,7 @@ int __init rds_ib_listen_init(void) goto out; } - rdsdebug("cm %p listening on port %u\n", cm_id, RDS_PORT); + rdsdebug("cm %p listening on port %u\n", cm_id, RDS_IB_PORT); rds_ib_listen_id = cm_id; cm_id = NULL; diff --git a/net/rds/rds.h b/net/rds/rds.h index 03031e2..6c17a4d 100644 --- a/net/rds/rds.h +++ b/net/rds/rds.h @@ -26,8 +26,8 @@ * * port 18633 was the version that had ack frames on the wire. */ -#define RDS_PORT 18634 - +#define RDS_TCP_PORT 18634 +#define RDS_IB_PORT 18635 #ifndef AF_RDS #define AF_RDS 28 /* Reliable Datagram Socket */ diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c index 0389a99..298e372 100644 --- a/net/rds/tcp_connect.c +++ b/net/rds/tcp_connect.c @@ -96,7 +96,7 @@ int rds_tcp_conn_connect(struct rds_connection *conn) dest.sin_family = AF_INET; dest.sin_addr.s_addr = (__force u32)conn->c_faddr; - dest.sin_port = (__force u16)htons(RDS_PORT); + dest.sin_port = (__force u16)htons(RDS_TCP_PORT); /* * once we call connect() we can start getting callbacks and they diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c index caeacbe..50709b7 100644 --- a/net/rds/tcp_listen.c +++ b/net/rds/tcp_listen.c @@ -159,7 +159,7 @@ int __init rds_tcp_listen_init(void) sin.sin_family = PF_INET, sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); - sin.sin_port = (__force u16)htons(RDS_PORT); + sin.sin_port = (__force u16)htons(RDS_TCP_PORT); ret = sock->ops->bind(sock, (struct sockaddr *)&sin, sizeof(sin)); if (ret < 0) From olaf.kirch at ORACLE.COM Tue Jul 8 13:55:00 2008 From: olaf.kirch at ORACLE.COM (Olaf Kirch) Date: Tue, 8 Jul 2008 22:55:00 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080708202755.GB31809@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807071440.46502.olaf.kirch@oracle.com> <20080708202755.GB31809@opengridcomputing.com> Message-ID: <200807082255.01460.olaf.kirch@oracle.com> Hi Jon, thanks for doing this! I'll merge everything tomorrow, and try to break it down into a series of nice patches, and it all goes well I will at least push the bcopy part to my 1.4 tree. > This patch reduces the default send_wr and recv_wr to sizes that Chelsio > can handle, as the sizes before were too large. Optimally, this should > be determined with a query of the device. I have added that to my todo > list as well Yes, that'll be nice to do properly. I'll think about this a little tomorrow. > The lguid and fguid cannot be used for iWARP as those structs do not > exists and give off a nasty illegal pointer deference. I removed the > references to those, as the appear to only be for debug messages. Fine with me. > Lastly, this has the port space differentiation (which is the only way > TCP and iWARP can happily co-exist). Okay. Olaf On Tuesday 08 July 2008 22:27:55 Jon Mason wrote: > Hey Olaf, > With the patch below on top of your previous patch, I am able to get it > working over iWARP (though RDMA is still breaking, but in a new and > different way). > > The patch modifies rds_ib_laddr_check to use rdma_bind_addr to determine > if it is a IB/iWARP device by creating a temporary rdma_cm_id and > binding to the address passed in. I'm sure this can be optimized, but > I'll leave that as a todo for later. > > This patch reduces the default send_wr and recv_wr to sizes that Chelsio > can handle, as the sizes before were too large. Optimally, this should > be determined with a query of the device. I have added that to my todo > list as well > > The lguid and fguid cannot be used for iWARP as those structs do not > exists and give off a nasty illegal pointer deference. I removed the > references to those, as the appear to only be for debug messages. > > Lastly, this has the port space differentiation (which is the only way > TCP and iWARP can happily co-exist). > > Let me know what you think. > > Thanks, > Jon > > Signed-off-by: Jon Mason > > diff --git a/net/rds/ib.c b/net/rds/ib.c > index cd2dc7c..067fab0 100644 > --- a/net/rds/ib.c > +++ b/net/rds/ib.c > @@ -158,17 +158,29 @@ void rds_ib_remove_one(struct ib_device *device) > */ > static int rds_ib_laddr_check(__be32 addr) > { > - struct net_device *dev; > - int ret; > + int ret = -EADDRNOTAVAIL; > + struct rdma_cm_id *cm_id; > + struct sockaddr_in sin; > + > + cm_id = rdma_create_id(NULL, NULL, RDMA_PS_TCP); > + if (cm_id == NULL) { > + printk("rdma_create_id failed\n"); > + return -EADDRNOTAVAIL; > + } > > - dev = ip_dev_find(addr); > - if (dev && dev->type == ARPHRD_INFINIBAND) { > - dev_put(dev); > - ret = 0; > - } else > + memset(&sin, 0, sizeof(sin)); > + sin.sin_family = AF_INET; > + sin.sin_addr.s_addr = addr; > + > + /* rdma_bind_addr will fail for non-IB/iWARP devices */ > + ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin); > + if (ret) > ret = -EADDRNOTAVAIL; > > - rdsdebug("addr %u.%u.%u.%u ret %d\n", NIPQUAD(addr), ret); > + rdsdebug("addr %u.%u.%u.%u ret %d node type %d\n", > + NIPQUAD(addr), ret, cm_id->device?cm_id->device->node_type:-1); > + > + rdma_destroy_id(cm_id); > > return ret; > } > diff --git a/net/rds/ib.h b/net/rds/ib.h > index 947977c..2924fda 100644 > --- a/net/rds/ib.h > +++ b/net/rds/ib.h > @@ -7,14 +7,16 @@ > > #define RDS_IB_RESOLVE_TIMEOUT_MS 5000 > > -#define RDS_FMR_SIZE 256 > +/* FIXME - use ib_query_device to determine proper value */ > +#define RDS_FMR_SIZE 20 > #define RDS_FMR_POOL_SIZE 2048 > > -#define RDS_IB_MAX_SGE 8 > +#define RDS_IB_MAX_SGE 8 > #define RDS_IB_RECV_SGE 2 > > -#define RDS_IB_DEFAULT_RECV_WR 1024 > -#define RDS_IB_DEFAULT_SEND_WR 256 > +/* FIXME - call ib_query_device to determine sane values for this based on the HW */ > +#define RDS_IB_DEFAULT_RECV_WR 512 > +#define RDS_IB_DEFAULT_SEND_WR 128 > > #define RDS_IB_SUPPORTED_PROTOCOLS 0x00000003 /* minor versions supported */ > > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > index a49e394..fd56481 100644 > --- a/net/rds/ib_cm.c > +++ b/net/rds/ib_cm.c > @@ -296,7 +296,7 @@ static int rds_ib_setup_qp(struct rds_connection *conn) > */ > ret = rdma_create_qp(ic->i_cm_id, ic->i_pd, &attr); > if (ret) { > - rdsdebug("ib_req_notify_cq failed: %d\n", ret); > + rdsdebug("rdma_create_qp failed: %d\n", ret); > goto out; > } > > @@ -388,8 +388,6 @@ static u32 rds_ib_protocol_compatible(const struct rds_ib_connect_private *dp) > static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, > struct rdma_cm_event *event) > { > - __be64 lguid = cm_id->route.path_rec->sgid.global.interface_id; > - __be64 fguid = cm_id->route.path_rec->dgid.global.interface_id; > const struct rds_ib_connect_private *dp = event->param.conn.private_data; > struct rds_ib_connect_private dp_rep; > struct rds_connection *conn = NULL; > @@ -404,11 +402,9 @@ static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, > if (!version) > goto out; > > - rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u lguid 0x%llx fguid " > - "0x%llx\n", NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), > - RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version), > - (unsigned long long)be64_to_cpu(lguid), > - (unsigned long long)be64_to_cpu(fguid)); > + rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u\n", > + NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), > + RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version)); > > conn = rds_conn_create(dp->dp_daddr, dp->dp_saddr, &rds_ib_transport, > GFP_KERNEL); > @@ -628,7 +624,7 @@ int rds_ib_conn_connect(struct rds_connection *conn) > > dest.sin_family = AF_INET; > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > - dest.sin_port = (__force u16)htons(RDS_PORT); > + dest.sin_port = (__force u16)htons(RDS_IB_PORT); > > ret = rdma_resolve_addr(ic->i_cm_id, (struct sockaddr *)&src, > (struct sockaddr *)&dest, > @@ -813,7 +809,7 @@ int __init rds_ib_listen_init(void) > > sin.sin_family = PF_INET, > sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); > - sin.sin_port = (__force u16)htons(RDS_PORT); > + sin.sin_port = (__force u16)htons(RDS_IB_PORT); > > /* > * XXX I bet this binds the cm_id to a device. If we want to support > @@ -833,7 +829,7 @@ int __init rds_ib_listen_init(void) > goto out; > } > > - rdsdebug("cm %p listening on port %u\n", cm_id, RDS_PORT); > + rdsdebug("cm %p listening on port %u\n", cm_id, RDS_IB_PORT); > > rds_ib_listen_id = cm_id; > cm_id = NULL; > diff --git a/net/rds/rds.h b/net/rds/rds.h > index 03031e2..6c17a4d 100644 > --- a/net/rds/rds.h > +++ b/net/rds/rds.h > @@ -26,8 +26,8 @@ > * > * port 18633 was the version that had ack frames on the wire. > */ > -#define RDS_PORT 18634 > - > +#define RDS_TCP_PORT 18634 > +#define RDS_IB_PORT 18635 > > #ifndef AF_RDS > #define AF_RDS 28 /* Reliable Datagram Socket */ > diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c > index 0389a99..298e372 100644 > --- a/net/rds/tcp_connect.c > +++ b/net/rds/tcp_connect.c > @@ -96,7 +96,7 @@ int rds_tcp_conn_connect(struct rds_connection *conn) > > dest.sin_family = AF_INET; > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > - dest.sin_port = (__force u16)htons(RDS_PORT); > + dest.sin_port = (__force u16)htons(RDS_TCP_PORT); > > /* > * once we call connect() we can start getting callbacks and they > diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c > index caeacbe..50709b7 100644 > --- a/net/rds/tcp_listen.c > +++ b/net/rds/tcp_listen.c > @@ -159,7 +159,7 @@ int __init rds_tcp_listen_init(void) > > sin.sin_family = PF_INET, > sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); > - sin.sin_port = (__force u16)htons(RDS_PORT); > + sin.sin_port = (__force u16)htons(RDS_TCP_PORT); > > ret = sock->ops->bind(sock, (struct sockaddr *)&sin, sizeof(sin)); > if (ret < 0) > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From rdreier at cisco.com Tue Jul 8 14:15:32 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 14:15:32 -0700 Subject: [ofa-general] Re: [PATCH v2] mlx4: make firmware diagnostic counters available via sysfs In-Reply-To: <200806181232.35634.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Wed, 18 Jun 2008 12:32:35 +0300") References: <200804021615.44982.jackm@dev.mellanox.co.il> <200806181232.35634.jackm@dev.mellanox.co.il> Message-ID: Looking at this in the context of what's queued already -- would it make sense to expose this via the protocol counters stuff that went in for iWARP? - R. From rdreier at cisco.com Tue Jul 8 14:21:34 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 14:21:34 -0700 Subject: [ofa-general] Re: [PATCH v2] mlx4: implement MOD_STAT_CFG command to use for changing ConnectX page size In-Reply-To: <4832C350.50004@dev.mellanox.co.il> (Vladimir Sokolovsky's message of "Tue, 20 May 2008 15:25:52 +0300") References: <4832C350.50004@dev.mellanox.co.il> Message-ID: thanks, applied. From rdreier at cisco.com Tue Jul 8 14:41:29 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 14:41:29 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get a fix for one regression introduced earlier in the 2.6.26 cycle: Steve Wise (1): RDMA/cxgb3: Fix regression caused by class_device -> device conversion drivers/infiniband/hw/cxgb3/iwch_provider.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index 8934178..95f82cf 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1096,7 +1096,9 @@ static ssize_t show_fw_ver(struct device *dev, struct device_attribute *attr, ch struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev; PDBG("%s dev 0x%p\n", __func__, dev); + rtnl_lock(); lldev->ethtool_ops->get_drvinfo(lldev, &info); + rtnl_unlock(); return sprintf(buf, "%s\n", info.fw_version); } @@ -1109,7 +1111,9 @@ static ssize_t show_hca(struct device *dev, struct device_attribute *attr, struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev; PDBG("%s dev 0x%p\n", __func__, dev); + rtnl_lock(); lldev->ethtool_ops->get_drvinfo(lldev, &info); + rtnl_unlock(); return sprintf(buf, "%s\n", info.driver); } From or.gerlitz at gmail.com Tue Jul 8 13:06:38 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 8 Jul 2008 23:06:38 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH v2 2/4] rdma/cma: simplify locking neededfor serialization of callbacks In-Reply-To: References: <000c01c8e12c$2ff11b00$a8c8180a@amr.corp.intel.com> <15ddcffd0807081259u47ed2e38i7388f9d38fd91f88@mail.gmail.com> Message-ID: <15ddcffd0807081306g77732d3ahf2e8466c6e98a139@mail.gmail.com> On 7/8/08, Roland Dreier wrote: > > After I read the code more closely I decided that the cma_enable_callback() > wrapper didn't make sense, because there are already so many places that > have to use the handler_mutex by hand. Oh, yes, that was another reason for not having the wrapper... So this is what I have committed: cool, thanks. Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From meier3 at llnl.gov Tue Jul 8 17:20:27 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Tue, 08 Jul 2008 17:20:27 -0700 Subject: [ofa-general] [PATCH] opensm: osm_subnet.c - added console loopback mode to opts file & parser Message-ID: <4874044B.50209@llnl.gov> Sasha, A small patch to save/restore the loopback mode for the console. -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing 925.422.3341 meier3 at llnl.gov -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-opensm-osm_subnet.c-added-console-loopback-mode-t.patch URL: From sashak at voltaire.com Tue Jul 8 18:07:35 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 04:07:35 +0300 Subject: [ofa-general] Re: [PATCH] opensm: osm_subnet.c - added console loopback mode to opts file & parser In-Reply-To: <4874044B.50209@llnl.gov> References: <4874044B.50209@llnl.gov> Message-ID: <20080709010734.GH26875@sashak.voltaire.com> On 17:20 Tue 08 Jul , Timothy A. Meier wrote: > From 96db02b70f4146ad190f655077136d7fdea37ebe Mon Sep 17 00:00:00 2001 > From: Tim Meier > Date: Tue, 8 Jul 2008 17:04:52 -0700 > Subject: [PATCH] opensm: osm_subnet.c - added console loopback mode to opts file & parser > > The loopback mode of the console (special socket case) has been available via > command line, and this patch provides support via the option caching > mechanism. > > Signed-off-by: Tim Meier Applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 8 18:09:30 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 04:09:30 +0300 Subject: [ofa-general] Re: [PATCH][TRIVIAL] infiniband-diags/mcm_rereg_test.c: Cosmetic formatting change In-Reply-To: <1215465178.18371.78.camel@hrosenstock-ws.xsigo.com> References: <1215465178.18371.78.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709010930.GI26875@sashak.voltaire.com> On 14:12 Mon 07 Jul , Hal Rosenstock wrote: > infiniband-diags/mcm_rereg_test.c: Cosmetic formatting change > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 8 18:09:45 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 04:09:45 +0300 Subject: [ofa-general] Re: [PATCH][TRIVIAL] libibmad/src/rpc.c: Eliminate compiler warning In-Reply-To: <1215464852.18371.70.camel@hrosenstock-ws.xsigo.com> References: <1215464852.18371.70.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709010945.GJ26875@sashak.voltaire.com> On 14:07 Mon 07 Jul , Hal Rosenstock wrote: > libibmad/src/rpc.c: Eliminate compiler warning > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 8 18:10:07 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 04:10:07 +0300 Subject: [ofa-general] Re: [PATCH] libibmad: Initial changes for congestion management class In-Reply-To: <1215465025.18371.74.camel@hrosenstock-ws.xsigo.com> References: <1215465025.18371.74.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709011007.GK26875@sashak.voltaire.com> On 14:10 Mon 07 Jul , Hal Rosenstock wrote: > libibmad: Initial changes for congestion management class > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 8 18:45:03 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 04:45:03 +0300 Subject: [ofa-general] Re: [OPENSM] [PATCH] reroute console option In-Reply-To: <1215465953.16508.58.camel@cardanus.llnl.gov> References: <1215465953.16508.58.camel@cardanus.llnl.gov> Message-ID: <20080709014503.GL26875@sashak.voltaire.com> Hi Al, On 14:25 Mon 07 Jul , Al Chu wrote: > > With all of these file-input possibilities for OpenSM routing, I don't > think there is a way to tell OpenSM to reroute if (for example) the file > input has changed but the network/fabric has not. Yes, but you can request a heavy sweep with SIGHUP or from console. > This is a patch to > add a command to the console to tell the console to reroute but not do a > heavy sweep. [snip...] > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > index 87d6d5b..0501aa2 100644 > --- a/opensm/opensm/osm_state_mgr.c > +++ b/opensm/opensm/osm_state_mgr.c > @@ -1020,6 +1020,7 @@ static void do_sweep(osm_sm_t * sm) > && sm->p_subn->sm_state != IB_SMINFO_STATE_DISCOVERING > && sm->p_subn->opt.force_heavy_sweep == FALSE > && sm->p_subn->force_heavy_sweep == FALSE > + && sm->p_subn->force_reroute == FALSE > && sm->p_subn->subnet_initialization_error == FALSE > && (__osm_state_mgr_light_sweep_start(sm) == IB_SUCCESS)) { > if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) > @@ -1031,11 +1032,43 @@ static void do_sweep(osm_sm_t * sm) > } > } > > + /* > + * If we don't need to do a heavy sweep and we want to do a reroute, > + * just reroute only. > + */ > + if (cl_qmap_count(&sm->p_subn->sw_guid_tbl) > + && sm->p_subn->sm_state != IB_SMINFO_STATE_DISCOVERING > + && sm->p_subn->opt.force_heavy_sweep == FALSE > + && sm->p_subn->force_heavy_sweep == FALSE > + && sm->p_subn->force_reroute == TRUE > + && sm->p_subn->subnet_initialization_error == FALSE) { > + /* Reset flag */ > + sm->p_subn->force_reroute = FALSE; > + > + /* Re-program the switches fully */ > + sm->p_subn->ignore_existing_lfts = TRUE; > + > + osm_ucast_mgr_process(&sm->ucast_mgr); > + > + /* Reset flag */ > + sm->p_subn->ignore_existing_lfts = FALSE; > + > + if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) > + return; > + > + if (!sm->p_subn->force_heavy_sweep) { I cannot see how 'force_heavy_sweep' can be triggered by osm_ucast_mgr_process(), but I do see that 'subnet_initialization_error' can be set to TRUE during re-routing. So it looks that we need to check this flag instead. Sasha > + osm_log_msg_box(sm->p_log, OSM_LOG_VERBOSE, > + __FUNCTION__, "REROUTE COMPLETE"); > + return; > + } > + } > + > /* go to heavy sweep */ > _repeat_discovery: > > /* First of all - unset all flags */ > sm->p_subn->force_heavy_sweep = FALSE; > + sm->p_subn->force_reroute = FALSE; > sm->p_subn->subnet_initialization_error = FALSE; > > /* rescan configuration updates */ > -- > 1.5.4.5 > From rdreier at cisco.com Tue Jul 8 19:58:18 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 08 Jul 2008 19:58:18 -0700 Subject: [ofa-general] [PATCH for-2.6.27 2/2] RDMA/cxgb3: Fixes for zero-stag. In-Reply-To: (Roland Dreier's message of "Tue, 08 Jul 2008 11:07:36 -0700") References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201710.20859.39200.stgit@dell3.ogc.int> Message-ID: by the way, this means we have new FW major versions for 2.6.25, 2.6.26 and 2.6.27. please, please can we get a libcxgb3 that works with FW 5 through 7 (since I don't think any of the changes affected userspace anyway, right)?? From sashak at voltaire.com Tue Jul 8 20:44:14 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 06:44:14 +0300 Subject: [ofa-general] Multicast traffic generates Bad P_Key trap in SM when working in partial member setup In-Reply-To: <1215533144.18371.109.camel@hrosenstock-ws.xsigo.com> References: <39C75744D164D948A170E9792AF8E7CA013E6E2B@exil.voltaire.com> <1213266626.14047.419.camel@hrosenstock-ws.xsigo.com> <1213270788.14047.451.camel@hrosenstock-ws.xsigo.com> <1213278134.14047.498.camel@hrosenstock-ws.xsigo.com> <1214509372.13056.645.camel@hrosenstock-ws.xsigo.com> <1215533144.18371.109.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709034414.GA12551@sashak.voltaire.com> Hi Hal, On 09:05 Tue 08 Jul , Hal Rosenstock wrote: > > Right, this trap is optional and I don't know whether or not it is > supported. Perhaps someone from Mellanox can comment on this to be sure. I looked at this switch and saw that IsPKeySwitchExternalPortTrapSupported bit in PortInfo:CapabilityMask at switch port 0 is not set. This means - no traps. > > The conclusion is that if there is switch enforcement the issue is not > > severe and we can live with it :) > > Would that be the case if those optional traps were supported ? Guess so. Sasha From sashak at voltaire.com Tue Jul 8 20:56:16 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 06:56:16 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> Message-ID: <20080709035616.GB12551@sashak.voltaire.com> Hi Tziporet, On 18:01 Sun 06 Jul , Tziporet Koren wrote: > > This is the list of libraries in OFED > We wish to define which are the libraries that are public and their API > is being preserved from release to release. I agree with this concept and suggest to keep only "public" and "others" lists. Actually to have only "public" list (libraries with stable API) should be enough. > There are libraries I already classified, and there are some I am not > sure about > Each owner - please reply regarding the libraries that are under unknown > Also - if I missed any library please add it to the list > > Thanks, > Tziporet > > Public: > ======= > * libibcommon I wanted to remove (merge with libibumad and libibmad) libibcommon yet in OFED 1.3 (we discussed this on the list). Not sure I will be able to do it now (in OFED 1.4 timeframe), but at least let's move it from "public" list. > * libdat2 > * libdat > * libibcm > * libibverbs > * libibmad > * libibumad > * libsdp > * librdmacm > > Private: > ======== > * libcxgb3 > * libehca > * libipathverbs > * libmlx4 > * libmthca > * libnes > > Unknown: > ======== > * libibdmcom > * libdaplcma > * libdaplofa > * libibdm > * libibis > * libibmscli > * libopensm > * libosmcomp > * libosmvendor > * libosmvendor_openib 'libosmvendor_openib' is not a library, just symbolic link. > * libumad2sim Sasha From sashak at voltaire.com Tue Jul 8 21:03:20 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 07:03:20 +0300 Subject: [ofa-general] Re: [PATCH RFC] opensm/event_plugin: plugin API version 2 In-Reply-To: <1215005835.3486.269.camel@hrosenstock-ws.xsigo.com> References: <1214501218.13056.620.camel@hrosenstock-ws.xsigo.com> <20080627181931.GO29456@sashak.voltaire.com> <20080627130333.08d783c6.weiny2@llnl.gov> <20080627203745.GA29198@obsidianresearch.com> <20080627230339.GV29456@sashak.voltaire.com> <1214825806.3486.173.camel@hrosenstock-ws.xsigo.com> <20080630142705.GF7229@sashak.voltaire.com> <1214861294.3486.216.camel@hrosenstock-ws.xsigo.com> <20080702131104.GI30305@sashak.voltaire.com> <1215005835.3486.269.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709040320.GC12551@sashak.voltaire.com> On 06:37 Wed 02 Jul , Hal Rosenstock wrote: > > Because exposing all the data structures, etc. is way more granular (all > the socalled opaque objects are no longer opaque) than a library API. > It's a totally different magnitude IMO. This is what I said. Having "structured" API could lead to an issues similar to what we saw with libosmcomp. > > > This is related to what I wrote below about the OpenFabrics licensing > > > requirements. The idea is if GPL licensing were to be allowed (perhaps > > > only in some limited context), then there could be two different > > > packages: dual and GPL. In that way plugins would be more assured of > > > being compatible with each other and OpenSM. > > > > I would prefer to separate packages by its functionality and not due to > > licensing issues. > > Right, licensed based packages do put all related functionality in a > similar bucket (management) but is that the basis to make such a > decision ? Which decisions? I'm not following. Sasha From sashak at voltaire.com Tue Jul 8 21:13:44 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 07:13:44 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: Only sendMCG delete notice if not previously done In-Reply-To: <26E4A768C32B0749989A2C07675EBABD1C3AF1@mtlexch01.mtl.com> References: <1215005954.3486.272.camel@hrosenstock-ws.xsigo.com> <26E4A768C32B0749989A2C07675EBABD1C3AF1@mtlexch01.mtl.com> Message-ID: <20080709041344.GD12551@sashak.voltaire.com> Hi Eitan, On 16:44 Wed 02 Jul , Eitan Zahavi wrote: > > Once a MCG group is deleted by the last member leaving the group (immediately) it > is required to still exist until the routing engine deletes its routing. Why MCG is required to still exist? MFT entries will be there until the routing engine rebuilds this (same is true for unicast). I cannot see any reason to keep deleted MCG. > We mark the group with "to be deleted" flag so we know it will be deleted soon. Correct and all "to be deleted" groups are ignored in all OpenSM operations. That was my original question - why we need to keep those "to be deleted" groups? Sasha From sashak at voltaire.com Tue Jul 8 22:03:03 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 08:03:03 +0300 Subject: [ofa-general] [ANNOUNCE] management tarballs release Message-ID: <20080709050303.GE12551@sashak.voltaire.com> Hi, There is a new release of the management (OpenSM and infiniband diagnostics) tarballs available in: http://www.openfabrics.org/downloads/management/ md5sum: 59737e8ef106c3a37a3bab690397d973 libibcommon-1.1.1.tar.gz 40d87f68e6259eb25e85f89d27b95f63 libibumad-1.2.1.tar.gz 891c907cf7fb56191c1cd4224608ef63 libibmad-1.2.1.tar.gz fd74b4456987ea78da8f1c2e7edf4f84 opensm-3.2.2.tar.gz ff809b62b2f6cead7d33fb722d8638b6 infiniband-diags-1.4.1.tar.gz All component versions are from recent master branch. Full change log is below. Sasha Al Chu (7): ids_guid_file manpage entry add guid_routing_order_file option rearch __osm_ucast_mgr_process_tbl() usage implement guid_routing_order_file fix true/false usage Fix comment typo Fix regenerate cache corner case. Hal Rosenstock (70): OpenSM: Add QoS_management_in_OpenSM.txt to opensm/doc directory OpenSM/doc/modular_routing.txt: Fix typo OpenSM/doc/QoS_management_in_OpenSM.txt: Remove mention of OFED management: Support separate SA and SM keys as clarified in IBA 1.2.1 opensm/osm_sa_mcmember_record.c: Improve log message and some comments relating to SNM opensm/main.c: Minor change to long option for consolidate_ipv6_snm_req OpenSM: Add another HP OUI to recognized vendor IDs opensm/osm_subnet.c: Change comment for IPv6 SNM in options file OpenSM/osm_sa_mcmember_record.c: Collapse all scopes when consolidating IPv6 SNM OpenSM release notes: Update to 3.1.11 opensm/osm_lid_mgr.c: Eliminate some potential NULL pointer dereferences opensm/osm_lin_fwd_tbl.c: Eliminate potential NULL pointer dereference OpenSM/osm_sa_mcmember_record.c: Validate some more MGID bits for IPv6 SNM infiniband-diags/saquery.c: Update for change to osm_mad_pool_init API opensm/osm_sa_mcmember_record.c: Minor logic change in __get_new_mlid opensm/osm_ucast_ftree.c: Eliminate unnecessary check in __osm_ftree_sw_tbl_element_create opensm/osm_pkey.c: Eliminate potential NULL pointer dereference opensm/osm_port.c: Eliminate potential NULL pointer dereferences libibmad/fields.c: _set_field64 sets field in network rather than host order infiniband-diags/mcm_rereg_test.c: Handle error when guid file not found infiniband-diags/ibsendtrap.c: Support CA and port num OpenSM/osm_sa_mcmember_record.c: Comment reformatting OpenSM: Remove some vestigial comments infiniband-diags/saquery.c: In dump_results, use query_result rather than query_svc_rec opensm/libvendor/osm_vendor_ibumad_sa.c: Cosmetic changes OpenSM/osm_sa_portinfo_record.c: Cosmetic comment change OpenSM/osm_sa_mcmember_record.c: Cosmetic change to error log message OpenSM/osm_sa_path_record.c: Add some information to some error log messages opensm/include/iba/ib_types.h: Fix comment OpenSM/include/opensm: Fix some commentary typos OpenSM/include/osm_port_profile.h: Fix some typos OpenSM/libvendor/osm_vendor_ibumad_sa.c: Eliminate unneeded define opensm/osm_lin_fwd_tbl.c: Minor change to __osm_lin_tbl_compute_obj_size infiniband-diags/saquery.c: In print_multicast_group_records, only query NodeRecords when needed opensm/osm_sa_path_record.c: Break up some long OSM_LOG lines opensm/osm_trap_rcv.c: Break up some long comment lines opensm/osm_sa_mcmember_record.c: Breakup some long lines opensm/osm_sa_mcmember_record.c: When consolidating SNM, need separate group per PKey opensm/man/opensm.8.in: Update consolidate_ipv6_snm_req description opensm/osm_subnet.c: Fix typo opensm/osm_sa_mcmember_record.c: Some error message improvements opensm/include/opensm/osm_multicast.h: Cosmetic changes opensm/osm_sa_mcmember_record.c: Cosmetic changes opensm/osm_mcast_mgr.c: Cosmetic comment format change opensm/osm_lid_mgr.c: Cosmetic formatting changes opensm/include/osm_port.h: Eliminate some unneeded includes opensm/include/osm_db.h: Fix some typos opensm/include/osm_prefix_route.h: Support C++ inclusion opensm/include/osm_mtree.h: Eliminate unneeded include libibmad/rpc.c: Better error message in madrpc_init opensm/osm_port.c: Cosmetic commentary changes opensm/osm_switch: Cosmetic formatting changes opensm/osm_subnet.h: Cosmetic formatting changes opensm/osm_sa_mcmember_record.c: Fix some off by 1 issues related to max_multicast_lid_ho opensm/osm_sw_info_rcv.c: Cosmetic formatting changes opensm/osm_lid_mgr.c: Fix off by 1 issue with max_lid opensm/include/complib/cl_fleximap.h: Improve cl_pfn_fmap_cmp_t PARAMETERS comments opensm/include/opensm/osm_port_profile.h: Cosmetic formatting changes opensm/osm_sw_info_rcv.c: Cosmetic formatting change opensm: Convert mgrp_mlid_tbl into array opensm/include/opensm/osm_port_profile.h: Cosmetic formatting changes opensm/include/opensm/osm_port.h: Commentary change osm_port_profile: Change some variable names for consistency opensm/man/opensm.8.in: Clarify ignore-guids file format opensm/osm_mcast_mgr.c: Only send MCG delete notice if not previously done opensm: Convert unicast LIDs in long messages from hex to decimal opensm/osm_port_profile: Handle all possible ports libibmad/src/rpc.c: Eliminate compiler warning libibmad: Initial changes for congestion management class infiniband-diags/mcm_rereg_test.c: Cosmetic formatting change Ira K. Weiny (2): libibmad: print warning containing destination port when _do_madrpc fails libibmad: update portid2str to use inet_ntop and drpath2str functions for gid Ira Weiny (11): opensm/opensm/osm_perfmgr.c: change log level of counter overflow message opensm/configure.in: Fix the QOS and prefix routes config file default locations infiniband-diags/scripts/iblinkinfo.pl: fix printing of switch name when port 1 is down. opensm/opensm/osm_lid_mgr.c: set "send_set" when setting rereg bit OpenSM: Fix rpm build, /opensm/opensm.conf failed to install OpenSM: Add a Performance Manager HOWTO to the docs and the dist infiniband-diags/scripts/ibprint[ca|switch|rt].pl: fix printing by name infiniband-diags/scripts/ibprint[ca|switch|rt].pl: allow printing of multiple matches but print warning to user that multiple matches were found infiniband-diags/src/ibnetdiscover.c: Fix the PortInfo data on the port we discover a switch on. infiniband-diags/src/saquery.c: fix potential core dump libibmad: add a drpath2str function for easier printing of DR Paths. Matthias Blankenhaus (1): saquery: port selection problem Max Matveev (1): libibmad: allow explicit HCA selection by functions from libibmad Ron Livne (1): libibmad: fix for umad unaligned access of ia64. Sasha Khapyorsky (75): opensm/osm_base.h: fix typo opensm/osm_lid_mgr.c: cosmetic opensm/doc/current-routing.txt: add DOR algorithm reference opensm: merge disable_multicast and no_multicast_option options opensm: remove unused pfn_ui_* callback options opensm: port_prof_ignore_file option opensm: write all OpenSM options to cache file opensm/scripts/opensm.init.in: fix status command opensm/scripts: remove not used opensmd template opensm/scripts: remove opensm.conf usage opensm: pass file name as parameter to config parser funcs opensm: config file functions return int opensm: option to specify config file opensm: use OSM_DEFAULT_CONFIG_FILE as config file infiniband-diags/scripts/*.pl: exit 2 on usage errors infiniband-diags/scripts/*.pl: prevent some zero exists on errors opensm/osm_lid_mgr.c: indentation fixes opensm: remove osm_log reference from osm_mad_pool object opensm: remove some vestigial comments and RCS keywords saquery: --smkey command line option opensm: improve log messages in __osm_state_mgr_check_tbl_consistency() infiniband-diags/man/saquery: add --smkey option to the man page opensm/configure.in: remove AM_MAINTAINER_MODE opensm: fix OSM_DEFAULT_SM_KEY byte order opensm: state default SM_Key value change in usage and man opensm: replace "(null)" string with null_str variable infiniband-diags/saquery: split print_multicast_group_records() function opensm: move remote guids counting to ucast_mgr opensm: preserve base lid routes opensm: consolidate config parser logging code opensm/osm_concole.c: trivial: break comment long lines opensm/osm_console.c: fix guid parsing opensm/inculde/*.h: remove empty lines in type definitions opensm/osm_console: use osm_get_switch_by_guid() opensm/osm_subnet.h: remove some not used typedefs osm_vendor_select.h: make OSM_VENDOR_INTF_OPENIB to be default opensm: rename 'struct _osm_*_t' to 'struct osm_*' opensm: replace inclusion of osm_vendor.h by osm_vendor_api.h opensm: make osm_perfmgr_db.* content dependent on PerfMgr availability opensm/osm_ucast_ftree.c: trivial: remove OSM_LOG_EXIT opensm/perfmgr: move event_db_dump_file to common OpenSM dump dir opensm/config: allow space separated strings as values opensm: move event_plugin from perfmgr to opensm opensm: support for multiple event plugins opensm: fix compilation error with --enable-debug cmoplib/*.h: trivial: remove empty lines in structure definitions opensm_ucast_mgr: add node name and port number to the logging opensm/osm_subnet: rename field mgrp_mlid_tbl to mgroups opensm/osm_sa_mcmember_record: cleanup __get_new_mlid() function opensm: merge all __get_mgrp_by_mlid() instances opensm: trivial rename max_unicast_lid_ho and max_multicast_lid_ho opensm/complib: cl_item_obj macro opensm/osm_ucast_mgr: some code consolidation and cleanup opensm/osm_subnet.h: comments formatting opensm: add --version command line option opensm/configure.in: fix HAVE_DEFAULT_OPENSM_CONFIG_FILE define generation opensm: --create-config command line option opensm/man: add -F. --config option to OpenSM man page opensm: workaround CL_ASSERT() failure with port_order_list qlist libibmad: remove intermediate buffer in portid2str() opensm/osm_log.h: trivial formatting in comments opensm: minor snprintf() usage improvements libibcommon: improve snprintf() usage libibumad: improve snprintf() usage. opensm: fix snprintf() usage libibumad: fix snprintf() usage infiniband-diags: fix snprintf() usage libibmad: fix snprintf() usage opensm/osm_sa_lft_record: trivial formatting fixes opensm: trivial formatting fixes opensm/osm_log: formatting changes management: revert configure.in update configure.in on 'make dist' management/make.dist: daily build updates management/make.dist: make branch checking unconditional management: bump management versions Tim Meier (1): opensm: osm_subnet.c - added console loopback mode to opts file & parser Vincent Ficet (1): opensm: fix wrong port_guid initialization Yevgeny Kliteynik (7): opensm/osm_state_mgr.c: fix segmentation fault infiniband-diags/Makefile.am: fix location of ibdiag_version.h opensm/ib_types.h: cosmetics - µsec t o usec opensm/ib_types.h: fixing some wrong comments opensm/osm_qos_parser.y: TRIVIAL addition in log message opensm/man/opensm.8.in: TRIVIAL addition in fat-tree description opensm/osm_state_mgr.c: fixing log message From eitan at mellanox.co.il Tue Jul 8 22:41:09 2008 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Wed, 9 Jul 2008 08:41:09 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: Only sendMCGdelete notice if not previously done In-Reply-To: <20080709041344.GD12551@sashak.voltaire.com> Message-ID: <40FA0A8088E8A441973D37502F00933E3F86FF@mtlexch01.mtl.com> The data in the old MCG struct is used for the actual delete. > -----Original Message----- > From: Sasha Khapyorsky [mailto:sashak at voltaire.com] > Sent: ד 09 יולי 2008 07:14 > To: Eitan Zahavi > Cc: Hal Rosenstock; general at lists.openfabrics.org > Subject: Re: [ofa-general] Re: [PATCH] > opensm/osm_mcast_mgr.c: Only sendMCGdelete notice if not > previously done > > Hi Eitan, > > On 16:44 Wed 02 Jul , Eitan Zahavi wrote: > > > > Once a MCG group is deleted by the last member leaving the group > > (immediately) it is required to still exist until the > routing engine deletes its routing. > > Why MCG is required to still exist? MFT entries will be there > until the routing engine rebuilds this (same is true for > unicast). I cannot see any reason to keep deleted MCG. > > > We mark the group with "to be deleted" flag so we know it > will be deleted soon. > > Correct and all "to be deleted" groups are ignored in all > OpenSM operations. That was my original question - why we > need to keep those "to be deleted" groups? > > Sasha > From tziporet at dev.mellanox.co.il Tue Jul 8 23:02:22 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Wed, 09 Jul 2008 09:02:22 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <20080709035616.GB12551@sashak.voltaire.com> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> Message-ID: <4874546E.8000108@mellanox.co.il> Sasha Khapyorsky wrote: > Hi Tziporet, > > On 18:01 Sun 06 Jul , Tziporet Koren wrote: > >> This is the list of libraries in OFED >> We wish to define which are the libraries that are public and their API >> is being preserved from release to release. >> > > I agree with this concept and suggest to keep only "public" and "others" > lists. Actually to have only "public" list (libraries with stable API) > should be enough. > > >> There are libraries I already classified, and there are some I am not >> sure about >> Each owner - please reply regarding the libraries that are under unknown >> Also - if I missed any library please add it to the list >> >> Thanks, >> Tziporet >> >> Public: >> ======= >> * libibcommon >> > > I wanted to remove (merge with libibumad and libibmad) libibcommon yet > in OFED 1.3 (we discussed this on the list). Not sure I will be able to > do it now (in OFED 1.4 timeframe), but at least let's move it from > "public" list. > > Has anyone using this library now? If not we can remove it to the private list >> * libdat2 >> * libdat >> * libibcm >> * libibverbs >> * libibmad >> * libibumad >> * libsdp >> * librdmacm >> >> Private: >> ======== >> * libcxgb3 >> * libehca >> * libipathverbs >> * libmlx4 >> * libmthca >> * libnes >> >> Unknown: >> ======== >> * libibdmcom >> * libdaplcma >> * libdaplofa >> * libibdm >> * libibis >> * libibmscli >> * libopensm >> * libosmcomp >> * libosmvendor >> > > 'libosmvendor_openib' is not a library, just symbolic link. > thanks > >> * libumad2sim >> From sashak at voltaire.com Tue Jul 8 23:03:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 09:03:05 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: Only sendMCGdelete notice if not previously done In-Reply-To: <40FA0A8088E8A441973D37502F00933E3F86FF@mtlexch01.mtl.com> References: <20080709041344.GD12551@sashak.voltaire.com> <40FA0A8088E8A441973D37502F00933E3F86FF@mtlexch01.mtl.com> Message-ID: <20080709060305.GH12551@sashak.voltaire.com> On 08:41 Wed 09 Jul , Eitan Zahavi wrote: > The data in the old MCG struct is used for the actual delete. Ok. I see. MFTs are cleared now on per-mlid basic. Likely it can be helpful to rework this part to be similar to unicast - build new MFT image, then send only changed blocks. And then we will not need to keep deleted MCG. Sasha From eitan at mellanox.co.il Tue Jul 8 23:15:02 2008 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Wed, 9 Jul 2008 09:15:02 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: OnlysendMCGdelete notice if not previously done In-Reply-To: <20080709060305.GH12551@sashak.voltaire.com> Message-ID: <40FA0A8088E8A441973D37502F00933E3F8717@mtlexch01.mtl.com> Sure. There are also some other aspects of MCG routing that can be improved ... > -----Original Message----- > From: Sasha Khapyorsky [mailto:sashak at voltaire.com] > Sent: ד 09 יולי 2008 09:03 > To: Eitan Zahavi > Cc: Hal Rosenstock; general at lists.openfabrics.org > Subject: Re: [ofa-general] Re: [PATCH] > opensm/osm_mcast_mgr.c: OnlysendMCGdelete notice if not > previously done > > On 08:41 Wed 09 Jul , Eitan Zahavi wrote: > > The data in the old MCG struct is used for the actual delete. > > Ok. I see. MFTs are cleared now on per-mlid basic. > > Likely it can be helpful to rework this part to be similar to > unicast - build new MFT image, then send only changed blocks. > And then we will not need to keep deleted MCG. > > Sasha > From sashak at voltaire.com Tue Jul 8 23:17:01 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 09:17:01 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <4874546E.8000108@mellanox.co.il> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> <4874546E.8000108@mellanox.co.il> Message-ID: <20080709061701.GI12551@sashak.voltaire.com> On 09:02 Wed 09 Jul , Tziporet Koren wrote: > Sasha Khapyorsky wrote: >> Hi Tziporet, >> >> On 18:01 Sun 06 Jul , Tziporet Koren wrote: >> >>> This is the list of libraries in OFED >>> We wish to define which are the libraries that are public and their API >>> is being preserved from release to release. >>> >> >> I agree with this concept and suggest to keep only "public" and "others" >> lists. Actually to have only "public" list (libraries with stable API) >> should be enough. >> >> >>> There are libraries I already classified, and there are some I am not >>> sure about >>> Each owner - please reply regarding the libraries that are under unknown >>> Also - if I missed any library please add it to the list >>> >>> Thanks, >>> Tziporet >>> >>> Public: >>> ======= >>> * libibcommon >>> >> >> I wanted to remove (merge with libibumad and libibmad) libibcommon yet >> in OFED 1.3 (we discussed this on the list). Not sure I will be able to >> do it now (in OFED 1.4 timeframe), but at least let's move it from >> "public" list. >> >> > Has anyone using this library now? Nobody directly (AFAIK), but via libibumad. I guess -libcommon is listed as part of LDFLAGS in some ibutils Makefile (this is why I didn't want to make a "noise" during 1.3 release cycle). > If not we can remove it to the private list I would suggest to not have a "private" list at all - only "public". Basically I see nothing wrong in using a "private" libraries by others. Just think that it should be clear which OFED libraries have stable API. Sasha From sashak at voltaire.com Tue Jul 8 23:18:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 09:18:08 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_mcast_mgr.c: OnlysendMCGdelete notice if not previously done In-Reply-To: <40FA0A8088E8A441973D37502F00933E3F8717@mtlexch01.mtl.com> References: <20080709060305.GH12551@sashak.voltaire.com> <40FA0A8088E8A441973D37502F00933E3F8717@mtlexch01.mtl.com> Message-ID: <20080709061808.GJ12551@sashak.voltaire.com> On 09:15 Wed 09 Jul , Eitan Zahavi wrote: > Sure. There are also some other aspects of MCG routing that can be improved ... Of course :). Thanks for explanations. Sasha From jackm at dev.mellanox.co.il Tue Jul 8 23:47:29 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 9 Jul 2008 09:47:29 +0300 Subject: [ofa-general] Re: [PATCH v2] mlx4: make firmware diagnostic counters available via sysfs In-Reply-To: References: <200804021615.44982.jackm@dev.mellanox.co.il> <200806181232.35634.jackm@dev.mellanox.co.il> Message-ID: <200807090947.29539.jackm@dev.mellanox.co.il> On Wednesday 09 July 2008 00:15, Roland Dreier wrote: > Looking at this in the context of what's queued already -- would it make > sense to expose this via the protocol counters stuff that went in for > iWARP? > > - R. The firmware diagnostic counters are not IB-protocol counters -- they are low-level device-specific (i.e., ConnectX-specific) counters. As such, I think the code to implement them belongs in the low-level driver layer. The problem here is that the sysfs counter file names are specified in the core layer (core/sysfs.c) via the IW_STATS_ENTRY macro at compilation time. If we are to implement exposing these firmware counters in the core layer, we need a low-level driver API which would allow the low-level driver to pass to the core layer at run time a vector of counter names, and we would then need to do all the compile-time operations (DEVICE_ATTR, declaration of all the "show functions", etc) at run time. It looks to me that this would be messy. For example, each "show" method is specific to one counter-name "file". We would then need to pre-declare (at compile time) a group of "show" methods whose names are the offsets, rather than the counter-name (e.g., device_name ## "counter-1", etc.), and this would have the effect of limiting the max number of counters we could generate. Despite the great code similarity, I think it is preferable to leave the firmware diagnostic counters in the mlx4 layer at this point. - Jack From jackm at dev.mellanox.co.il Wed Jul 9 00:25:47 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 9 Jul 2008 10:25:47 +0300 Subject: [ofa-general] [PATCH 1/11] libibverbs: XRC base implementation In-Reply-To: References: <200806231601.19743.jackm@dev.mellanox.co.il> Message-ID: <200807091025.48022.jackm@dev.mellanox.co.il> On Monday 23 June 2008 19:33, Roland Dreier wrote: > > +struct ibv_srq *__ibv_create_xrc_srq(struct ibv_pd *pd, > > > +default_symver(__ibv_create_xrc_srq, ibv_create_xrc_srq); > > Am I missing something? Why do we need this default_symver() stuff when > we're adding new functions that didn't exist in the 1.0 ABI? > No, you're not missing anything. My mistake (I mistakenly used a function which did exist in the 1.0 ABI as my model). - Jack From vlad at lists.openfabrics.org Wed Jul 9 02:39:49 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Wed, 9 Jul 2008 02:39:49 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080709-0200 daily build status Message-ID: <20080709093949.521CFE6095B@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080709-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From tziporet at dev.mellanox.co.il Wed Jul 9 02:42:22 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Wed, 09 Jul 2008 12:42:22 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <20080709061701.GI12551@sashak.voltaire.com> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> <4874546E.8000108@mellanox.co.il> <20080709061701.GI12551@sashak.voltaire.com> Message-ID: <487487FE.1080406@mellanox.co.il> Sasha Khapyorsky wrote: > On 09:02 Wed 09 Jul , Tziporet Koren wrote: > >>>> Public: >>>> ======= >>>> * libibcommon >>>> >>>> >>> I wanted to remove (merge with libibumad and libibmad) libibcommon yet >>> in OFED 1.3 (we discussed this on the list). Not sure I will be able to >>> do it now (in OFED 1.4 timeframe), but at least let's move it from >>> "public" list. >>> >>> >>> >> Has anyone using this library now? >> > > Nobody directly (AFAIK), but via libibumad. I guess -libcommon is listed > as part of LDFLAGS in some ibutils Makefile (this is why I didn't want > to make a "noise" during 1.3 release cycle). > Oren - are you using libibcommon in ibutils? Tziporet From eli at mellanox.co.il Wed Jul 9 02:48:31 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 9 Jul 2008 12:48:31 +0300 Subject: [ofa-general] [PATCH] IB/mlx4: Use kzalloc when allocating a new mlx4_ib_qp Message-ID: <20080709094830.GA3915@mtls03> Current code uses kmalloc() and then does a bitwise or operation on qp->flags at create_qp_common(): if (init_attr->create_flags & IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; Instead of taking care of clearing individual fields, just use kzalloc and forget about similar problems in the future. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/qp.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 91590e7..f4fbd5c 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -704,7 +704,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, case IB_QPT_UC: case IB_QPT_UD: { - qp = kmalloc(sizeof *qp, GFP_KERNEL); + qp = kzalloc(sizeof *qp, GFP_KERNEL); if (!qp) return ERR_PTR(-ENOMEM); @@ -725,7 +725,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, if (pd->uobject) return ERR_PTR(-EINVAL); - sqp = kmalloc(sizeof *sqp, GFP_KERNEL); + sqp = kzalloc(sizeof *sqp, GFP_KERNEL); if (!sqp) return ERR_PTR(-ENOMEM); -- 1.5.6 From orenk at dev.mellanox.co.il Wed Jul 9 03:31:35 2008 From: orenk at dev.mellanox.co.il (Oren Kladnitsky) Date: Wed, 09 Jul 2008 13:31:35 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <487487FE.1080406@mellanox.co.il> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> <4874546E.8000108@mellanox.co.il> <20080709061701.GI12551@sashak.voltaire.com> <487487FE.1080406@mellanox.co.il> Message-ID: <48749387.2060706@dev.mellanox.co.il> Tziporet Koren wrote: > Sasha Khapyorsky wrote: >> On 09:02 Wed 09 Jul , Tziporet Koren wrote: >> >>>>> Public: >>>>> ======= >>>>> * libibcommon >>>>> >>>> I wanted to remove (merge with libibumad and libibmad) libibcommon yet >>>> in OFED 1.3 (we discussed this on the list). Not sure I will be >>>> able to >>>> do it now (in OFED 1.4 timeframe), but at least let's move it from >>>> "public" list. >>>> >>>> >>> Has anyone using this library now? >>> >> >> Nobody directly (AFAIK), but via libibumad. I guess -libcommon is listed >> as part of LDFLAGS in some ibutils Makefile (this is why I didn't want >> to make a "noise" during 1.3 release cycle). >> > Oren - are you using libibcommon in ibutils? > > Tziporet > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg No. Ibutils use opensm libvendor for mad sending interface. From yevgenyp at mellanox.co.il Wed Jul 9 06:34:51 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:34:51 +0300 Subject: [ofa-general][PATCH 0/11 v3] mlx4: Multi Protocol support. Message-ID: <4874BE7B.9030008@mellanox.co.il> Hello Roland, This is the third round for patches for mlx4 VPI (Multi Protocol) support. The patches include changes that were made according to the feedback that we received from you, and also some additional features. The main changes/additions in this series of patches are: 1. Using log values for numeric module parameters. 2. In our bitmap allocator, the table now includes only ranges that can be allocated, not including the ranges reserved from top. 3. The Mac and Vlan tables are being initialized through port_info initialization. 4. Changed the dynamic port configuration mechanism, to work per port. We use a separate sysfs file for each port. 5. Added port type auto negotiation support. 6. Added support for additional device IDs. Thanks, Yevgeny From yevgenyp at mellanox.co.il Wed Jul 9 06:34:57 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:34:57 +0300 Subject: [ofa-general][PATCH 1/11 v3] mlx4: Qp range reservation Message-ID: <4874BE81.6060600@mellanox.co.il> mlx4: Qp range reservation Prior to allocating a qp, one need to reserve an aligned range of qps. The change is made to enable allocation of consecutive qps. Diff from previous version: - mlx4_bitmap_free() uses mlx4_bitmap_free_range() with range=1. - Free qp range if failed to allocate qp. Signed-off-by: Yevgeny Petrilin --- drivers/infiniband/hw/mlx4/qp.c | 13 ++++++- drivers/net/mlx4/alloc.c | 73 +++++++++++++++++++++++++++++++++++++- drivers/net/mlx4/mlx4.h | 2 + drivers/net/mlx4/qp.c | 44 ++++++++++++++++------- include/linux/mlx4/device.h | 5 ++- 5 files changed, 120 insertions(+), 17 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 4b0ac5d..e3b44d0 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -549,10 +549,17 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } } - err = mlx4_qp_alloc(dev->dev, sqpn, &qp->mqp); + if (!sqpn) + err = mlx4_qp_reserve_range(dev->dev, 1, 1, &sqpn); if (err) goto err_wrid; + err = mlx4_qp_alloc(dev->dev, sqpn, &qp->mqp); + if (err) { + mlx4_qp_release_range(dev->dev, sqpn, 1); + goto err_wrid; + } + /* * Hardware wants QPN written in big-endian order (after * shifting) for send doorbell. Precompute this value to save @@ -659,6 +666,10 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp, mlx4_ib_unlock_cqs(send_cq, recv_cq); mlx4_qp_free(dev->dev, &qp->mqp); + + if (!is_sqp(dev, qp)) + mlx4_qp_release_range(dev->dev, qp->mqp.qpn, 1); + mlx4_mtt_cleanup(dev->dev, &qp->mtt); if (is_user) { diff --git a/drivers/net/mlx4/alloc.c b/drivers/net/mlx4/alloc.c index f9d6b4d..e9b97e3 100644 --- a/drivers/net/mlx4/alloc.c +++ b/drivers/net/mlx4/alloc.c @@ -64,16 +64,85 @@ u32 mlx4_bitmap_alloc(struct mlx4_bitmap *bitmap) void mlx4_bitmap_free(struct mlx4_bitmap *bitmap, u32 obj) { + mlx4_bitmap_free_range(bitmap, obj, 1); +} + +static unsigned long find_aligned_range(unsigned long *bitmap, + u32 start, u32 nbits, + int len, int align) +{ + unsigned long end, i; + +again: + start = ALIGN(start, align); + while ((start < nbits) && test_bit(start, bitmap)) + start += align; + if (start >= nbits) + return -1; + + end = start+len; + if (end > nbits) + return -1; + for (i = start+1; i < end; i++) { + if (test_bit(i, bitmap)) { + start = i+1; + goto again; + } + } + return start; +} + +u32 mlx4_bitmap_alloc_range(struct mlx4_bitmap *bitmap, int cnt, int align) +{ + u32 obj, i; + + if (likely(cnt == 1 && align == 1)) + return mlx4_bitmap_alloc(bitmap); + + spin_lock(&bitmap->lock); + + obj = find_aligned_range(bitmap->table, bitmap->last, + bitmap->max, cnt, align); + if (obj >= bitmap->max) { + bitmap->top = (bitmap->top + bitmap->max) & bitmap->mask; + obj = find_aligned_range(bitmap->table, 0, + bitmap->max, + cnt, align); + } + + if (obj < bitmap->max) { + for (i = 0; i < cnt; i++) + set_bit(obj+i, bitmap->table); + if (obj == bitmap->last) { + bitmap->last = (obj + cnt); + if (bitmap->last >= bitmap->max) + bitmap->last = 0; + } + obj |= bitmap->top; + } else + obj = -1; + + spin_unlock(&bitmap->lock); + + return obj; +} + +void mlx4_bitmap_free_range(struct mlx4_bitmap *bitmap, u32 obj, int cnt) +{ + u32 i; + obj &= bitmap->max - 1; spin_lock(&bitmap->lock); - clear_bit(obj, bitmap->table); + for (i = 0; i < cnt; i++) + clear_bit(obj+i, bitmap->table); bitmap->last = min(bitmap->last, obj); bitmap->top = (bitmap->top + bitmap->max) & bitmap->mask; spin_unlock(&bitmap->lock); } -int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask, u32 reserved) +int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, + u32 num, u32 mask, u32 reserved) { int i; diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a4023c2..89d4ccc 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -287,6 +287,8 @@ static inline struct mlx4_priv *mlx4_priv(struct mlx4_dev *dev) u32 mlx4_bitmap_alloc(struct mlx4_bitmap *bitmap); void mlx4_bitmap_free(struct mlx4_bitmap *bitmap, u32 obj); +u32 mlx4_bitmap_alloc_range(struct mlx4_bitmap *bitmap, int cnt, int align); +void mlx4_bitmap_free_range(struct mlx4_bitmap *bitmap, u32 obj, int cnt); int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask, u32 reserved); void mlx4_bitmap_cleanup(struct mlx4_bitmap *bitmap); diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c index ee5484c..f4a6dc5 100644 --- a/drivers/net/mlx4/qp.c +++ b/drivers/net/mlx4/qp.c @@ -147,19 +147,42 @@ int mlx4_qp_modify(struct mlx4_dev *dev, struct mlx4_mtt *mtt, } EXPORT_SYMBOL_GPL(mlx4_qp_modify); -int mlx4_qp_alloc(struct mlx4_dev *dev, int sqpn, struct mlx4_qp *qp) +int mlx4_qp_reserve_range(struct mlx4_dev *dev, int cnt, int align, int *base) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + struct mlx4_qp_table *qp_table = &priv->qp_table; + int qpn; + + qpn = mlx4_bitmap_alloc_range(&qp_table->bitmap, cnt, align); + if (qpn == -1) + return -ENOMEM; + + *base = qpn; + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_qp_reserve_range); + +void mlx4_qp_release_range(struct mlx4_dev *dev, int base_qpn, int cnt) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + struct mlx4_qp_table *qp_table = &priv->qp_table; + if (base_qpn < dev->caps.sqp_start + 8) + return; + + mlx4_bitmap_free_range(&qp_table->bitmap, base_qpn, cnt); +} +EXPORT_SYMBOL_GPL(mlx4_qp_release_range); + +int mlx4_qp_alloc(struct mlx4_dev *dev, int qpn, struct mlx4_qp *qp) { struct mlx4_priv *priv = mlx4_priv(dev); struct mlx4_qp_table *qp_table = &priv->qp_table; int err; - if (sqpn) - qp->qpn = sqpn; - else { - qp->qpn = mlx4_bitmap_alloc(&qp_table->bitmap); - if (qp->qpn == -1) - return -ENOMEM; - } + if (!qpn) + return -EINVAL; + + qp->qpn = qpn; err = mlx4_table_get(dev, &qp_table->qp_table, qp->qpn); if (err) @@ -208,9 +231,6 @@ err_put_qp: mlx4_table_put(dev, &qp_table->qp_table, qp->qpn); err_out: - if (!sqpn) - mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); - return err; } EXPORT_SYMBOL_GPL(mlx4_qp_alloc); @@ -240,8 +260,6 @@ void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp) mlx4_table_put(dev, &qp_table->auxc_table, qp->qpn); mlx4_table_put(dev, &qp_table->qp_table, qp->qpn); - if (qp->qpn >= dev->caps.sqp_start + 8) - mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); } EXPORT_SYMBOL_GPL(mlx4_qp_free); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index a744383..7136b16 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -386,7 +386,10 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, int collapsed); void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq); -int mlx4_qp_alloc(struct mlx4_dev *dev, int sqpn, struct mlx4_qp *qp); +int mlx4_qp_reserve_range(struct mlx4_dev *dev, int cnt, int align, int *base); +void mlx4_qp_release_range(struct mlx4_dev *dev, int base_qpn, int cnt); + +int mlx4_qp_alloc(struct mlx4_dev *dev, int qpn, struct mlx4_qp *qp); void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp); int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, struct mlx4_mtt *mtt, -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:08 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:08 +0300 Subject: [ofa-general][PATCH 3/11 v3] mlx4: Different port type support Message-ID: <4874BE8C.5000807@mellanox.co.il> [PATCH] mlx4: Different port type support Multiprotocol supports different port types. The port types are delivered through module parameters, crossed with firmware capabilities. Each consumer of mlx4_core should query for supported port types, mlx4_ib can no longer assume that all physical ports belong to it. Signed-off-by: Yevgeny Petrilin --- drivers/infiniband/hw/mlx4/mad.c | 6 +- drivers/infiniband/hw/mlx4/main.c | 12 ++++- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 + drivers/net/mlx4/fw.c | 4 ++ drivers/net/mlx4/fw.h | 1 + drivers/net/mlx4/main.c | 82 ++++++++++++++++++++++++++++++++++ include/linux/mlx4/device.h | 32 +++++++++++++ 7 files changed, 134 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c index 4c1e72f..d91ba56 100644 --- a/drivers/infiniband/hw/mlx4/mad.c +++ b/drivers/infiniband/hw/mlx4/mad.c @@ -297,7 +297,7 @@ int mlx4_ib_mad_init(struct mlx4_ib_dev *dev) int p, q; int ret; - for (p = 0; p < dev->dev->caps.num_ports; ++p) + for (p = 0; p < dev->num_ports; ++p) for (q = 0; q <= 1; ++q) { agent = ib_register_mad_agent(&dev->ib_dev, p + 1, q ? IB_QPT_GSI : IB_QPT_SMI, @@ -313,7 +313,7 @@ int mlx4_ib_mad_init(struct mlx4_ib_dev *dev) return 0; err: - for (p = 0; p < dev->dev->caps.num_ports; ++p) + for (p = 0; p < dev->num_ports; ++p) for (q = 0; q <= 1; ++q) if (dev->send_agent[p][q]) ib_unregister_mad_agent(dev->send_agent[p][q]); @@ -326,7 +326,7 @@ void mlx4_ib_mad_cleanup(struct mlx4_ib_dev *dev) struct ib_mad_agent *agent; int p, q; - for (p = 0; p < dev->dev->caps.num_ports; ++p) { + for (p = 0; p < dev->num_ports; ++p) { for (q = 0; q <= 1; ++q) { agent = dev->send_agent[p][q]; dev->send_agent[p][q] = NULL; diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 4d61e32..098dcd2 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -558,11 +558,15 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) MLX4_INIT_DOORBELL_LOCK(&ibdev->uar_lock); ibdev->dev = dev; + ibdev->ports_map = mlx4_get_ports_of_type(dev, MLX4_PORT_TYPE_IB); strlcpy(ibdev->ib_dev.name, "mlx4_%d", IB_DEVICE_NAME_MAX); ibdev->ib_dev.owner = THIS_MODULE; ibdev->ib_dev.node_type = RDMA_NODE_IB_CA; - ibdev->ib_dev.phys_port_cnt = dev->caps.num_ports; + ibdev->num_ports = 0; + mlx4_foreach_port(i, ibdev->ports_map) + ibdev->num_ports++; + ibdev->ib_dev.phys_port_cnt = ibdev->num_ports; ibdev->ib_dev.num_comp_vectors = 1; ibdev->ib_dev.dma_device = &dev->pdev->dev; @@ -676,7 +680,7 @@ static void mlx4_ib_remove(struct mlx4_dev *dev, void *ibdev_ptr) struct mlx4_ib_dev *ibdev = ibdev_ptr; int p; - for (p = 1; p <= dev->caps.num_ports; ++p) + for (p = 1; p <= ibdev->num_ports; ++p) mlx4_CLOSE_PORT(dev, p); mlx4_ib_mad_cleanup(ibdev); @@ -691,6 +695,10 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr, enum mlx4_dev_event event, int port) { struct ib_event ibev; + struct mlx4_ib_dev *ibdev = to_mdev((struct ib_device *) ibdev_ptr); + + if (port > ibdev->num_ports) + return; switch (event) { case MLX4_DEV_EVENT_PORT_UP: diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 5cf9947..9d4f7a7 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -155,6 +155,8 @@ struct mlx4_ib_ah { struct mlx4_ib_dev { struct ib_device ib_dev; struct mlx4_dev *dev; + u32 ports_map; + int num_ports; void __iomem *uar_map; struct mlx4_uar priv_uar; diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index b0ad0d1..e875b08 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -322,6 +322,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_pkeys[i] = 1 << (field & 0xf); } } else { +#define QUERY_PORT_SUPPORTED_TYPE_OFFSET 0x00 #define QUERY_PORT_MTU_OFFSET 0x01 #define QUERY_PORT_WIDTH_OFFSET 0x06 #define QUERY_PORT_MAX_GID_PKEY_OFFSET 0x07 @@ -334,6 +335,9 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) if (err) goto out; + MLX4_GET(field, outbox, + QUERY_PORT_SUPPORTED_TYPE_OFFSET); + dev_cap->supported_port_types[i] = field & 3; MLX4_GET(field, outbox, QUERY_PORT_MTU_OFFSET); dev_cap->max_mtu[i] = field & 0xf; MLX4_GET(field, outbox, QUERY_PORT_WIDTH_OFFSET); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index a2e827c..50a6a7d 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -97,6 +97,7 @@ struct mlx4_dev_cap { u32 reserved_lkey; u64 max_icm_sz; int max_gso_sz; + u8 supported_port_types[MLX4_MAX_PORTS + 1]; u8 log_max_macs[MLX4_MAX_PORTS + 1]; u8 log_max_vlans[MLX4_MAX_PORTS + 1]; }; diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 6d8daf4..0f52e1a 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -98,11 +98,48 @@ module_param_named(use_prio, use_prio, bool, 0444); MODULE_PARM_DESC(use_prio, "Enable steering by VLAN priority on ETH ports " "(0/1, default 0)"); +static char *port_type_arr[MLX4_MAX_PORTS] = { [0 ... (MLX4_MAX_PORTS-1)] = "ib"}; +module_param_array_named(port_type, port_type_arr, charp, NULL, 0444); +MODULE_PARM_DESC(port_type, "Ports L2 type (ib/eth/auto, entry per port, " + "comma seperated, default ib for all)"); + +static int mlx4_check_port_params(struct mlx4_dev *dev, + enum mlx4_port_type *port_type) +{ + if (port_type[0] != port_type[1] && + !(dev->caps.flags & MLX4_DEV_CAP_FLAG_DPDP)) { + mlx4_err(dev, "Only same port types supported " + "on this HCA, aborting.\n"); + return -EINVAL; + } + if ((port_type[0] == MLX4_PORT_TYPE_ETH) && + (port_type[1] == MLX4_PORT_TYPE_IB)) { + mlx4_err(dev, "eth-ib configuration is not supported.\n"); + return -EINVAL; + } + return 0; +} + +static void mlx4_str2port_type(char **port_str, + enum mlx4_port_type *port_type) +{ + int i; + + for (i = 0; i < MLX4_MAX_PORTS; i++) { + if (!strcmp(port_str[i], "eth")) + port_type[i] = MLX4_PORT_TYPE_ETH; + else + port_type[i] = MLX4_PORT_TYPE_IB; + } +} static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) { int err; int i; + enum mlx4_port_type port_type[MLX4_MAX_PORTS]; + + mlx4_str2port_type(port_type_arr, port_type); err = mlx4_QUERY_DEV_CAP(dev, dev_cap); if (err) { @@ -178,7 +215,24 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.log_num_vlans = log_num_vlan; dev->caps.log_num_prios = use_prio ? 3: 0; + err = mlx4_check_port_params(dev, port_type); + if (err) + return err; + for (i = 1; i <= dev->caps.num_ports; ++i) { + if (!dev_cap->supported_port_types[i]) { + mlx4_warn(dev, "FW doesn't support Multi Protocol, " + "loading IB only\n"); + dev->caps.port_type[i] = MLX4_PORT_TYPE_IB; + continue; + } + if (port_type[i-1] & dev_cap->supported_port_types[i]) + dev->caps.port_type[i] = port_type[i-1]; + else { + mlx4_err(dev, "Requested port type for port %d " + "not supported by HW\n", i); + return -ENODEV; + } if (dev->caps.log_num_macs > dev_cap->log_max_macs[i]) { dev->caps.log_num_macs = dev_cap->log_max_macs[i]; mlx4_warn(dev, "Requested number of MACs is too much " @@ -1002,10 +1056,38 @@ static struct pci_driver mlx4_driver = { .remove = __devexit_p(mlx4_remove_one) }; +static int __init mlx4_verify_params(void) +{ + int i; + + for (i = 0; i < MLX4_MAX_PORTS; ++i) { + if (strcmp(port_type_arr[i], "eth") && + strcmp(port_type_arr[i], "ib")) { + printk(KERN_WARNING "mlx4_core: bad port_type for " + "port %d: %s\n", i, port_type_arr[i]); + return -1; + } + } + if ((log_num_mac < 0) || (log_num_mac > 7)) { + printk(KERN_WARNING "mlx4_core: bad num_mac: %d\n", log_num_mac); + return -1; + } + + if ((log_num_vlan < 0) || (log_num_vlan > 7)) { + printk(KERN_WARNING "mlx4_core: bad num_vlan: %d\n", log_num_vlan); + return -1; + } + + return 0; +} + static int __init mlx4_init(void) { int ret; + if (mlx4_verify_params()) + return -EINVAL; + ret = mlx4_catas_init(); if (ret) return ret; diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 2935a21..ec60b7b 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -60,6 +60,7 @@ enum { MLX4_DEV_CAP_FLAG_IPOIB_CSUM = 1 << 7, MLX4_DEV_CAP_FLAG_BAD_PKEY_CNTR = 1 << 8, MLX4_DEV_CAP_FLAG_BAD_QKEY_CNTR = 1 << 9, + MLX4_DEV_CAP_FLAG_DPDP = 1 << 12, MLX4_DEV_CAP_FLAG_MEM_WINDOW = 1 << 16, MLX4_DEV_CAP_FLAG_APM = 1 << 17, MLX4_DEV_CAP_FLAG_ATOMIC = 1 << 18, @@ -141,6 +142,11 @@ enum qp_region { MLX4_QP_REGION_COUNT }; +enum mlx4_port_type { + MLX4_PORT_TYPE_IB = 1 << 0, + MLX4_PORT_TYPE_ETH = 1 << 1, +}; + enum { MLX4_NUM_FEXCH = 64 * 1024, }; @@ -204,6 +210,7 @@ struct mlx4_caps { int log_num_macs; int log_num_vlans; int log_num_prios; + enum mlx4_port_type port_type[MLX4_MAX_PORTS + 1]; }; struct mlx4_buf_list { @@ -358,6 +365,31 @@ struct mlx4_init_port_param { u64 si_guid; }; +static inline void mlx4_query_steer_cap(struct mlx4_dev *dev, int *log_mac, + int *log_vlan, int *log_prio) +{ + *log_mac = dev->caps.log_num_macs; + *log_vlan = dev->caps.log_num_vlans; + *log_prio = dev->caps.log_num_prios; +} + +static inline u32 mlx4_get_ports_of_type(struct mlx4_dev *dev, + enum mlx4_port_type ptype) +{ + u32 ret = 0; + int i; + + for (i = 1; i <= dev->caps.num_ports; ++i) { + if (dev->caps.port_type[i] == ptype) + ret |= 1 << (i-1); + } + return ret; +} + +#define mlx4_foreach_port(port, bitmap) \ + for ((port) = 1; (port) <= MLX4_MAX_PORTS; (port)++) \ + if (bitmap & 1 << ((port)-1)) + int mlx4_buf_alloc(struct mlx4_dev *dev, int size, int max_direct, struct mlx4_buf *buf); void mlx4_buf_free(struct mlx4_dev *dev, int size, struct mlx4_buf *buf); -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:19 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:19 +0300 Subject: [ofa-general][PATCH 5/11 v3] mlx4: Mac Vlan Management Message-ID: <4874BE97.1000701@mellanox.co.il> mlx4: Mac Vlan Management mlx4_core is now responsible for managing Mac and Vlan filters for each port. It also notifies the FW which port type will be loaded, using the SET_PORT command Signed-off-by: Yevgeny Petrilin Diff from previous version: - port_info structures are listed from 1 - mac and vlan tables are initialized through port_info initialization --- drivers/net/mlx4/Makefile | 2 +- drivers/net/mlx4/main.c | 27 +++++ drivers/net/mlx4/mlx4.h | 37 ++++++ drivers/net/mlx4/port.c | 273 +++++++++++++++++++++++++++++++++++++++++++ include/linux/mlx4/cmd.h | 9 ++ include/linux/mlx4/device.h | 6 + 6 files changed, 353 insertions(+), 1 deletions(-) create mode 100644 drivers/net/mlx4/port.c diff --git a/drivers/net/mlx4/Makefile b/drivers/net/mlx4/Makefile index 0952a65..f4932d8 100644 --- a/drivers/net/mlx4/Makefile +++ b/drivers/net/mlx4/Makefile @@ -1,4 +1,4 @@ obj-$(CONFIG_MLX4_CORE) += mlx4_core.o mlx4_core-y := alloc.o catas.o cmd.o cq.o eq.o fw.o icm.o intf.o main.o mcg.o \ - mr.o pd.o profile.o qp.o reset.o srq.o + mr.o pd.o profile.o qp.o reset.o srq.o port.o diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 531add4..e6be948 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -663,6 +663,7 @@ static int mlx4_setup_hca(struct mlx4_dev *dev) { struct mlx4_priv *priv = mlx4_priv(dev); int err; + int port; err = mlx4_init_uar_table(dev); if (err) { @@ -761,8 +762,20 @@ static int mlx4_setup_hca(struct mlx4_dev *dev) goto err_qp_table_free; } + for (port = 1; port <= dev->caps.num_ports; port++) { + err = mlx4_SET_PORT(dev, port); + if (err) { + mlx4_err(dev, "Failed to set port %d, aborting\n", + port); + goto err_mcg_table_free; + } + } + return 0; +err_mcg_table_free: + mlx4_cleanup_mcg_table(dev); + err_qp_table_free: mlx4_cleanup_qp_table(dev); @@ -826,11 +839,22 @@ no_msi: priv->eq_table.eq[i].irq = dev->pdev->irq; } +static void mlx4_init_port_info(struct mlx4_dev *dev, int port) +{ + struct mlx4_port_info *info = &mlx4_priv(dev)->port[port]; + + info->dev = dev; + info->port = port; + mlx4_init_mac_table(dev, &info->mac_table); + mlx4_init_vlan_table(dev, &info->vlan_table); +} + static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) { struct mlx4_priv *priv; struct mlx4_dev *dev; int err; + int port; printk(KERN_INFO PFX "Initializing %s\n", pci_name(pdev)); @@ -940,6 +964,9 @@ static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) if (err) goto err_close; + for (port = 1; port <= dev->caps.num_ports; port++) + mlx4_init_port_info(dev, port); + err = mlx4_register_device(dev); if (err) goto err_cleanup; diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a9b3ead..a90a28c 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -251,6 +251,37 @@ struct mlx4_catas_err { struct list_head list; }; +struct mlx4_mac_table { +#define MLX4_MAX_MAC_NUM 128 +#define MLX4_MAC_MASK 0xffffffffffff +#define MLX4_MAC_VALID_SHIFT 63 +#define MLX4_MAC_TABLE_SIZE (MLX4_MAX_MAC_NUM << 3) + __be64 entries[MLX4_MAX_MAC_NUM]; + int refs[MLX4_MAX_MAC_NUM]; + struct semaphore mac_sem; + int total; + int max; +}; + +struct mlx4_vlan_table { +#define MLX4_MAX_VLAN_NUM 126 +#define MLX4_VLAN_MASK 0xfff +#define MLX4_VLAN_VALID (1 << 31) +#define MLX4_VLAN_TABLE_SIZE (MLX4_MAX_VLAN_NUM << 2) + __be32 entries[MLX4_MAX_VLAN_NUM]; + int refs[MLX4_MAX_VLAN_NUM]; + struct semaphore vlan_sem; + int total; + int max; +}; + +struct mlx4_port_info { + struct mlx4_dev *dev; + int port; + struct mlx4_mac_table mac_table; + struct mlx4_vlan_table vlan_table; +}; + struct mlx4_priv { struct mlx4_dev dev; @@ -279,6 +310,7 @@ struct mlx4_priv { struct mlx4_uar driver_uar; void __iomem *kar; + struct mlx4_port_info port[MLX4_MAX_PORTS + 1]; }; static inline struct mlx4_priv *mlx4_priv(struct mlx4_dev *dev) @@ -349,4 +381,9 @@ void mlx4_srq_event(struct mlx4_dev *dev, u32 srqn, int event_type); void mlx4_handle_catas_err(struct mlx4_dev *dev); +void mlx4_init_mac_table(struct mlx4_dev *dev, struct mlx4_mac_table *table); +void mlx4_init_vlan_table(struct mlx4_dev *dev, struct mlx4_vlan_table *table); + +int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port); + #endif /* MLX4_H */ diff --git a/drivers/net/mlx4/port.c b/drivers/net/mlx4/port.c new file mode 100644 index 0000000..321d024 --- /dev/null +++ b/drivers/net/mlx4/port.c @@ -0,0 +1,273 @@ +/* + * Copyright (c) 2007 Mellanox Technologies. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#include +#include + +#include + +#include "mlx4.h" + +void mlx4_init_mac_table(struct mlx4_dev *dev, struct mlx4_mac_table *table) +{ + int i; + + sema_init(&table->mac_sem, 1); + for (i = 0; i < MLX4_MAX_MAC_NUM; i++) { + table->entries[i] = 0; + table->refs[i] = 0; + } + table->max = 1 << dev->caps.log_num_macs; + table->total = 0; +} + +void mlx4_init_vlan_table(struct mlx4_dev *dev, struct mlx4_vlan_table *table) +{ + int i; + + sema_init(&table->vlan_sem, 1); + for (i = 0; i < MLX4_MAX_MAC_NUM; i++) { + table->entries[i] = 0; + table->refs[i] = 0; + } + table->max = 1 << dev->caps.log_num_vlans; + table->total = 0; +} + +static int mlx4_SET_PORT_mac_table(struct mlx4_dev *dev, u8 port, + __be64 *entries) +{ + struct mlx4_cmd_mailbox *mailbox; + u32 in_mod; + int err; + + mailbox = mlx4_alloc_cmd_mailbox(dev); + if (IS_ERR(mailbox)) + return PTR_ERR(mailbox); + + memcpy(mailbox->buf, entries, MLX4_MAC_TABLE_SIZE); + + in_mod = MLX4_SET_PORT_MAC_TABLE << 8 | port; + err = mlx4_cmd(dev, mailbox->dma, in_mod, 1, MLX4_CMD_SET_PORT, + MLX4_CMD_TIME_CLASS_B); + + mlx4_free_cmd_mailbox(dev, mailbox); + return err; +} + +int mlx4_register_mac(struct mlx4_dev *dev, u8 port, u64 mac, int *index) +{ + struct mlx4_mac_table *table = &mlx4_priv(dev)->port[port].mac_table; + int i, err = 0; + int free = -1; + u64 valid = 1; + + mlx4_dbg(dev, "Registering mac : 0x%llx\n", mac); + down(&table->mac_sem); + for (i = 0; i < MLX4_MAX_MAC_NUM - 1; i++) { + if (free < 0 && !table->refs[i]) { + free = i; + continue; + } + + if (mac == (MLX4_MAC_MASK & be64_to_cpu(table->entries[i]))) { + /* Mac already registered, increase refernce count */ + *index = i; + ++table->refs[i]; + goto out; + } + } + mlx4_dbg(dev, "Free mac index is %d\n", free); + + if (table->total == table->max) { + /* No free mac entries */ + err = -ENOSPC; + goto out; + } + + /* Register new MAC */ + table->refs[free] = 1; + table->entries[free] = cpu_to_be64(mac | valid << MLX4_MAC_VALID_SHIFT); + + err = mlx4_SET_PORT_mac_table(dev, port, table->entries); + if (unlikely(err)) { + mlx4_err(dev, "Failed adding mac: 0x%llx\n", mac); + table->refs[free] = 0; + table->entries[free] = 0; + goto out; + } + + *index = free; + ++table->total; +out: + up(&table->mac_sem); + return err; +} +EXPORT_SYMBOL_GPL(mlx4_register_mac); + +void mlx4_unregister_mac(struct mlx4_dev *dev, u8 port, int index) +{ + struct mlx4_mac_table *table = &mlx4_priv(dev)->port[port].mac_table; + + down(&table->mac_sem); + if (!table->refs[index]) { + mlx4_warn(dev, "No mac entry for index %d\n", index); + goto out; + } + if (--table->refs[index]) { + mlx4_warn(dev, "Have more references for index %d," + "no need to modify mac table\n", index); + goto out; + } + table->entries[index] = 0; + mlx4_SET_PORT_mac_table(dev, port, table->entries); + --table->total; +out: + up(&table->mac_sem); +} +EXPORT_SYMBOL_GPL(mlx4_unregister_mac); + +static int mlx4_SET_PORT_vlan_table(struct mlx4_dev *dev, u8 port, + __be32 *entries) +{ + struct mlx4_cmd_mailbox *mailbox; + u32 in_mod; + int err; + + mailbox = mlx4_alloc_cmd_mailbox(dev); + if (IS_ERR(mailbox)) + return PTR_ERR(mailbox); + + memcpy(mailbox->buf, entries, MLX4_VLAN_TABLE_SIZE); + in_mod = MLX4_SET_PORT_VLAN_TABLE << 8 | port; + err = mlx4_cmd(dev, mailbox->dma, in_mod, 1, MLX4_CMD_SET_PORT, + MLX4_CMD_TIME_CLASS_B); + + mlx4_free_cmd_mailbox(dev, mailbox); + + return err; +} + +int mlx4_register_vlan(struct mlx4_dev *dev, u8 port, u16 vlan, int *index) +{ + struct mlx4_vlan_table *table = &mlx4_priv(dev)->port[port].vlan_table; + int i, err = 0; + int free = -1; + + down(&table->vlan_sem); + for (i = 0; i < MLX4_MAX_VLAN_NUM; i++) { + if (free < 0 && (table->refs[i] == 0)) { + free = i; + continue; + } + + if (table->refs[i] && + (vlan == (MLX4_VLAN_MASK & + be32_to_cpu(table->entries[i])))) { + /* Vlan already registered, increase refernce count */ + *index = i; + ++table->refs[i]; + goto out; + } + } + + if (table->total == table->max) { + /* No free vlan entries */ + err = -ENOSPC; + goto out; + } + + /* Register new MAC */ + table->refs[free] = 1; + table->entries[free] = cpu_to_be32(vlan | MLX4_VLAN_VALID); + + err = mlx4_SET_PORT_vlan_table(dev, port, table->entries); + if (unlikely(err)) { + mlx4_warn(dev, "Failed adding vlan: %u\n", vlan); + table->refs[free] = 0; + table->entries[free] = 0; + goto out; + } + + *index = free; + ++table->total; +out: + up(&table->vlan_sem); + return err; +} +EXPORT_SYMBOL_GPL(mlx4_register_vlan); + +void mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, int index) +{ + struct mlx4_vlan_table *table = &mlx4_priv(dev)->port[port].vlan_table; + + down(&table->vlan_sem); + if (!table->refs[index]) { + mlx4_warn(dev, "No vlan entry for index %d\n", index); + goto out; + } + if (--table->refs[index]) { + mlx4_dbg(dev, "Have more references for index %d," + "no need to modify vlan table\n", index); + goto out; + } + table->entries[index] = 0; + mlx4_SET_PORT_vlan_table(dev, port, table->entries); + --table->total; +out: + up(&table->vlan_sem); +} +EXPORT_SYMBOL_GPL(mlx4_unregister_vlan); + +int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port) +{ + struct mlx4_cmd_mailbox *mailbox; + int err; + u8 is_eth = (dev->caps.port_type[port] == MLX4_PORT_TYPE_ETH) ? 1 : 0; + + mailbox = mlx4_alloc_cmd_mailbox(dev); + if (IS_ERR(mailbox)) + return PTR_ERR(mailbox); + + memset(mailbox->buf, 0, 256); + if (is_eth) { + ((u8 *) mailbox->buf)[3] = 6; + ((__be16 *) mailbox->buf)[4] = cpu_to_be16(1 << 15); + ((__be16 *) mailbox->buf)[6] = cpu_to_be16(1 << 15); + } + err = mlx4_cmd(dev, mailbox->dma, port, is_eth, MLX4_CMD_SET_PORT, + MLX4_CMD_TIME_CLASS_B); + + mlx4_free_cmd_mailbox(dev, mailbox); + return err; +} diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h index 77323a7..cf9c679 100644 --- a/include/linux/mlx4/cmd.h +++ b/include/linux/mlx4/cmd.h @@ -132,6 +132,15 @@ enum { MLX4_MAILBOX_SIZE = 4096 }; +enum { + /* set port opcode modifiers */ + MLX4_SET_PORT_GENERAL = 0x0, + MLX4_SET_PORT_RQP_CALC = 0x1, + MLX4_SET_PORT_MAC_TABLE = 0x2, + MLX4_SET_PORT_VLAN_TABLE = 0x3, + MLX4_SET_PORT_PRIO_MAP = 0x4, +}; + struct mlx4_dev; struct mlx4_cmd_mailbox { diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index a076b59..d97314d 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -455,6 +455,12 @@ int mlx4_CLOSE_PORT(struct mlx4_dev *dev, int port); int mlx4_multicast_attach(struct mlx4_dev *dev, struct mlx4_qp *qp, u8 gid[16]); int mlx4_multicast_detach(struct mlx4_dev *dev, struct mlx4_qp *qp, u8 gid[16]); +int mlx4_register_mac(struct mlx4_dev *dev, u8 port, u64 mac, int *index); +void mlx4_unregister_mac(struct mlx4_dev *dev, u8 port, int index); + +int mlx4_register_vlan(struct mlx4_dev *dev, u8 port, u16 vlan, int *index); +void mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, int index); + int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, int npages, u64 iova, u32 *lkey, u32 *rkey); int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:28 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:28 +0300 Subject: [ofa-general][PATCH 6/11 v3] mlx4: Dynamic port configuration Message-ID: <4874BEA0.8040009@mellanox.co.il> mlx4: Dynamic port configuration Port type can be set using sysfs interface when the low level driver is up. The low level driver unregisters all its customers and then registers them again with the new port types (which they query for in add_one) Signed-off-by: Yevgeny Petrilin Diff from previous version: -Using a separate sysfs file per port for the port type configuration --- drivers/net/mlx4/main.c | 132 ++++++++++++++++++++++++++++++++++++++++++++-- drivers/net/mlx4/mlx4.h | 4 ++ 2 files changed, 130 insertions(+), 6 deletions(-) diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index e6be948..4da4682 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -266,6 +266,92 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) return 0; } +/* Changes the port configuration of the device. + * Every user of this function must hold the port lock */ +static int mlx4_change_port_types(struct mlx4_dev *dev, + enum mlx4_port_type *port_types) +{ + int err = 0; + int change = 0; + int port; + + for (port = 0; port < MLX4_MAX_PORTS; port++) { + if (port_types[port] != dev->caps.port_type[port + 1]) { + change = 1; + dev->caps.port_type[port + 1] = port_types[port]; + } + } + if (change) { + mlx4_unregister_device(dev); + for (port = 1; port <= dev->caps.num_ports; port++) { + mlx4_CLOSE_PORT(dev, port); + err = mlx4_SET_PORT(dev, port); + if (err) { + mlx4_err(dev, "Failed to set port %d, " + "aborting\n", port); + goto out; + } + } + err = mlx4_register_device(dev); + } + +out: + return err; +} + +static ssize_t show_port_type(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct mlx4_port_info *info = container_of(attr, struct mlx4_port_info, + port_attr); + struct mlx4_dev *mdev = info->dev; + + sprintf(buf, "%s\n", (mdev->caps.port_type[info->port] == MLX4_PORT_TYPE_IB)? + "ib": "eth"); + return strlen(buf); +} + +static ssize_t set_port_type(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct mlx4_port_info *info = container_of(attr, struct mlx4_port_info, + port_attr); + struct mlx4_dev *mdev = info->dev; + struct mlx4_priv *priv = mlx4_priv(mdev); + enum mlx4_port_type types[MLX4_MAX_PORTS]; + int i; + int err = 0; + + if (!strcmp(buf, "ib\n")) + info->tmp_type = MLX4_PORT_TYPE_IB; + else if (!strcmp(buf, "eth\n")) + info->tmp_type = MLX4_PORT_TYPE_ETH; + else { + mlx4_err(mdev, "%s is not supported port type\n", buf); + return -EINVAL; + } + + spin_lock(&priv->port_lock); + for (i = 0; i < mdev->caps.num_ports; i++) + types[i] = priv->port[i+1].tmp_type ? priv->port[i+1].tmp_type: + mdev->caps.port_type[i+1]; + + err = mlx4_check_port_params(mdev, types); + if (err) + goto out; + + for (i = 1; i <= mdev->caps.num_ports; i++) + priv->port[i].tmp_type = 0; + + err = mlx4_change_port_types(mdev, types); + +out: + spin_unlock(&priv->port_lock); + return err? err: count; +} + static int mlx4_load_fw(struct mlx4_dev *dev) { struct mlx4_priv *priv = mlx4_priv(dev); @@ -839,14 +925,38 @@ no_msi: priv->eq_table.eq[i].irq = dev->pdev->irq; } -static void mlx4_init_port_info(struct mlx4_dev *dev, int port) +static int mlx4_init_port_info(struct mlx4_dev *dev, int port) { struct mlx4_port_info *info = &mlx4_priv(dev)->port[port]; + struct attribute attr = {.name = info->dev_name, + .mode = S_IWUGO | S_IRUGO}; + int err = 0; info->dev = dev; info->port = port; mlx4_init_mac_table(dev, &info->mac_table); mlx4_init_vlan_table(dev, &info->vlan_table); + + sprintf(info->dev_name, "mlx4_port%d", port); + memcpy(&info->port_attr.attr, &attr, sizeof(attr)); + info->port_attr.show = show_port_type; + info->port_attr.store = set_port_type; + + err = device_create_file(&dev->pdev->dev, &info->port_attr); + if (err) { + mlx4_err(dev, "Failed to create file for port %d\n", port); + info->port = -1; + } + + return err; +} + +static void mlx4_cleanup_port_info(struct mlx4_port_info *info) +{ + if (info->port < 0) + return; + + device_remove_file(&info->dev->pdev->dev, &info->port_attr); } static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) @@ -929,6 +1039,8 @@ static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) INIT_LIST_HEAD(&priv->ctx_list); spin_lock_init(&priv->ctx_lock); + spin_lock_init(&priv->port_lock); + INIT_LIST_HEAD(&priv->pgdir_list); mutex_init(&priv->pgdir_mutex); @@ -964,18 +1076,24 @@ static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) if (err) goto err_close; - for (port = 1; port <= dev->caps.num_ports; port++) - mlx4_init_port_info(dev, port); + for (port = 1; port <= dev->caps.num_ports; port++) { + err = mlx4_init_port_info(dev, port); + if (err) + goto err_port; + } err = mlx4_register_device(dev); if (err) - goto err_cleanup; + goto err_port; pci_set_drvdata(pdev, dev); return 0; -err_cleanup: +err_port: + for (port = 1; port <= dev->caps.num_ports; port++) + mlx4_cleanup_port_info(&priv->port[port]); + mlx4_cleanup_mcg_table(dev); mlx4_cleanup_qp_table(dev); mlx4_cleanup_srq_table(dev); @@ -1032,8 +1150,10 @@ static void mlx4_remove_one(struct pci_dev *pdev) if (dev) { mlx4_unregister_device(dev); - for (p = 1; p <= dev->caps.num_ports; ++p) + for (p = 1; p <= dev->caps.num_ports; p++) { + mlx4_cleanup_port_info(&priv->port[p]); mlx4_CLOSE_PORT(dev, p); + } mlx4_cleanup_mcg_table(dev); mlx4_cleanup_qp_table(dev); diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a90a28c..a6205c5 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -278,6 +278,9 @@ struct mlx4_vlan_table { struct mlx4_port_info { struct mlx4_dev *dev; int port; + char dev_name[16]; + struct device_attribute port_attr; + enum mlx4_port_type tmp_type; struct mlx4_mac_table mac_table; struct mlx4_vlan_table vlan_table; }; @@ -311,6 +314,7 @@ struct mlx4_priv { struct mlx4_uar driver_uar; void __iomem *kar; struct mlx4_port_info port[MLX4_MAX_PORTS + 1]; + spinlock_t port_lock; }; static inline struct mlx4_priv *mlx4_priv(struct mlx4_dev *dev) -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:01 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:01 +0300 Subject: [ofa-general][PATCH 2/11 v3] mlx4: Pre reserved Qp regions. Message-ID: <4874BE85.6070108@mellanox.co.il> mlx4: Pre reserved Qp regions. We reserve Qp ranges to be used by other modules in case the ports come up as Ethernet ports. The qps are reserved at the end of the QP table. (This way we assure that they are aligned to their size) We need to consider these reserved ranges in bitmap creation : The reserved_top parameter. Diff from previous version: - Using Log base 2 for max mac and vlan numbers - Allocating only required size in bitmap allocation, without the bits reserved from top Signed-off-by: Yevgeny Petrilin --- drivers/net/mlx4/alloc.c | 30 ++++++++++++-------- drivers/net/mlx4/cq.c | 2 +- drivers/net/mlx4/eq.c | 2 +- drivers/net/mlx4/fw.c | 5 +++ drivers/net/mlx4/fw.h | 2 + drivers/net/mlx4/main.c | 63 ++++++++++++++++++++++++++++++++++++++---- drivers/net/mlx4/mcg.c | 4 +- drivers/net/mlx4/mlx4.h | 4 ++- drivers/net/mlx4/mr.c | 2 +- drivers/net/mlx4/pd.c | 4 +- drivers/net/mlx4/qp.c | 50 ++++++++++++++++++++++++++++++++- drivers/net/mlx4/srq.c | 2 +- include/linux/mlx4/device.h | 19 ++++++++++++- include/linux/mlx4/qp.h | 4 +++ 14 files changed, 163 insertions(+), 30 deletions(-) diff --git a/drivers/net/mlx4/alloc.c b/drivers/net/mlx4/alloc.c index e9b97e3..3571c1e 100644 --- a/drivers/net/mlx4/alloc.c +++ b/drivers/net/mlx4/alloc.c @@ -46,13 +46,16 @@ u32 mlx4_bitmap_alloc(struct mlx4_bitmap *bitmap) obj = find_next_zero_bit(bitmap->table, bitmap->max, bitmap->last); if (obj >= bitmap->max) { - bitmap->top = (bitmap->top + bitmap->max) & bitmap->mask; + bitmap->top = (bitmap->top + bitmap->max + bitmap->reserved_top) + & bitmap->mask; obj = find_first_zero_bit(bitmap->table, bitmap->max); } if (obj < bitmap->max) { set_bit(obj, bitmap->table); - bitmap->last = (obj + 1) & (bitmap->max - 1); + bitmap->last = (obj + 1); + if (bitmap->last == bitmap->max) + bitmap->last = 0; obj |= bitmap->top; } else obj = -1; @@ -104,9 +107,9 @@ u32 mlx4_bitmap_alloc_range(struct mlx4_bitmap *bitmap, int cnt, int align) obj = find_aligned_range(bitmap->table, bitmap->last, bitmap->max, cnt, align); if (obj >= bitmap->max) { - bitmap->top = (bitmap->top + bitmap->max) & bitmap->mask; - obj = find_aligned_range(bitmap->table, 0, - bitmap->max, + bitmap->top = (bitmap->top + bitmap->max + bitmap->reserved_top) + & bitmap->mask; + obj = find_aligned_range(bitmap->table, 0, bitmap->max, cnt, align); } @@ -131,18 +134,19 @@ void mlx4_bitmap_free_range(struct mlx4_bitmap *bitmap, u32 obj, int cnt) { u32 i; - obj &= bitmap->max - 1; + obj &= bitmap->max + bitmap->reserved_top - 1; spin_lock(&bitmap->lock); for (i = 0; i < cnt; i++) clear_bit(obj+i, bitmap->table); bitmap->last = min(bitmap->last, obj); - bitmap->top = (bitmap->top + bitmap->max) & bitmap->mask; + bitmap->top = (bitmap->top + bitmap->max + bitmap->reserved_top) + & bitmap->mask; spin_unlock(&bitmap->lock); } -int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, - u32 num, u32 mask, u32 reserved) +int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask, + u32 reserved_bot, u32 reserved_top) { int i; @@ -152,14 +156,16 @@ int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, bitmap->last = 0; bitmap->top = 0; - bitmap->max = num; + bitmap->max = num - reserved_top; bitmap->mask = mask; + bitmap->reserved_top = reserved_top; spin_lock_init(&bitmap->lock); - bitmap->table = kzalloc(BITS_TO_LONGS(num) * sizeof (long), GFP_KERNEL); + bitmap->table = kzalloc(BITS_TO_LONGS(bitmap->max) * + sizeof (long), GFP_KERNEL); if (!bitmap->table) return -ENOMEM; - for (i = 0; i < reserved; ++i) + for (i = 0; i < reserved_bot; ++i) set_bit(i, bitmap->table); return 0; diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 95e87a2..471e54a 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -300,7 +300,7 @@ int mlx4_init_cq_table(struct mlx4_dev *dev) INIT_RADIX_TREE(&cq_table->tree, GFP_ATOMIC); err = mlx4_bitmap_init(&cq_table->bitmap, dev->caps.num_cqs, - dev->caps.num_cqs - 1, dev->caps.reserved_cqs); + dev->caps.num_cqs - 1, dev->caps.reserved_cqs, 0); if (err) return err; diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c index e141a15..35c37a9 100644 --- a/drivers/net/mlx4/eq.c +++ b/drivers/net/mlx4/eq.c @@ -557,7 +557,7 @@ int mlx4_init_eq_table(struct mlx4_dev *dev) int i; err = mlx4_bitmap_init(&priv->eq_table.bitmap, dev->caps.num_eqs, - dev->caps.num_eqs - 1, dev->caps.reserved_eqs); + dev->caps.num_eqs - 1, dev->caps.reserved_eqs, 0); if (err) return err; diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index d82f275..b0ad0d1 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -325,6 +325,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_PORT_MTU_OFFSET 0x01 #define QUERY_PORT_WIDTH_OFFSET 0x06 #define QUERY_PORT_MAX_GID_PKEY_OFFSET 0x07 +#define QUERY_PORT_MAX_MACVLAN_OFFSET 0x0a #define QUERY_PORT_MAX_VL_OFFSET 0x0b for (i = 1; i <= dev_cap->num_ports; ++i) { @@ -342,6 +343,10 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_pkeys[i] = 1 << (field & 0xf); MLX4_GET(field, outbox, QUERY_PORT_MAX_VL_OFFSET); dev_cap->max_vl[i] = field & 0xf; + MLX4_GET(field, outbox, QUERY_PORT_MAX_MACVLAN_OFFSET); + dev_cap->log_max_macs[i] = field & 0xf; + dev_cap->log_max_vlans[i] = field >> 4; + } } diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index 306cb9b..a2e827c 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -97,6 +97,8 @@ struct mlx4_dev_cap { u32 reserved_lkey; u64 max_icm_sz; int max_gso_sz; + u8 log_max_macs[MLX4_MAX_PORTS + 1]; + u8 log_max_vlans[MLX4_MAX_PORTS + 1]; }; struct mlx4_adapter { diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index a6aa49f..6d8daf4 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -85,6 +85,20 @@ static struct mlx4_profile default_profile = { .num_mtt = 1 << 20, }; +static int log_num_mac = 2; +module_param_named(log_num_mac, log_num_mac, int, 0444); +MODULE_PARM_DESC(log_num_mac, "Log 2 Max number of MACs per ETH port (1-7)"); + +static int log_num_vlan; +module_param_named(log_num_vlan, log_num_vlan, int, 0444); +MODULE_PARM_DESC(log_num_vlan, "Log 2 Max number of VLANs per ETH port (0-7)"); + +static int use_prio; +module_param_named(use_prio, use_prio, bool, 0444); +MODULE_PARM_DESC(use_prio, "Enable steering by VLAN priority on ETH ports " + "(0/1, default 0)"); + + static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) { int err; @@ -134,7 +148,6 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.max_rq_sg = dev_cap->max_rq_sg; dev->caps.max_wqes = dev_cap->max_qp_sz; dev->caps.max_qp_init_rdma = dev_cap->max_requester_per_qp; - dev->caps.reserved_qps = dev_cap->reserved_qps; dev->caps.max_srq_wqes = dev_cap->max_srq_sz; dev->caps.max_srq_sge = dev_cap->max_rq_sg - 1; dev->caps.reserved_srqs = dev_cap->reserved_srqs; @@ -161,6 +174,39 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.stat_rate_support = dev_cap->stat_rate_support; dev->caps.max_gso_sz = dev_cap->max_gso_sz; + dev->caps.log_num_macs = log_num_mac; + dev->caps.log_num_vlans = log_num_vlan; + dev->caps.log_num_prios = use_prio ? 3: 0; + + for (i = 1; i <= dev->caps.num_ports; ++i) { + if (dev->caps.log_num_macs > dev_cap->log_max_macs[i]) { + dev->caps.log_num_macs = dev_cap->log_max_macs[i]; + mlx4_warn(dev, "Requested number of MACs is too much " + "for port %d, reducing to %d.\n", + i, 1 << dev->caps.log_num_macs); + } + if (dev->caps.log_num_vlans > dev_cap->log_max_vlans[i]) { + dev->caps.log_num_vlans = dev_cap->log_max_vlans[i]; + mlx4_warn(dev, "Requested number of VLANs is too much " + "for port %d, reducing to %d.\n", + i, 1 << dev->caps.log_num_vlans); + } + } + + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW] = dev_cap->reserved_qps; + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_ETH_ADDR] = + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FC_ADDR] = + (1 << dev->caps.log_num_macs)* + (1 << dev->caps.log_num_vlans)* + (1 << dev->caps.log_num_prios)* + dev->caps.num_ports; + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FC_EXCH] = MLX4_NUM_FEXCH; + + dev->caps.reserved_qps = dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW] + + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_ETH_ADDR] + + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FC_EXCH] + + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FC_EXCH]; + return 0; } @@ -209,7 +255,8 @@ static int mlx4_init_cmpt_table(struct mlx4_dev *dev, u64 cmpt_base, ((u64) (MLX4_CMPT_TYPE_QP * cmpt_entry_sz) << MLX4_CMPT_SHIFT), cmpt_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0, 0); + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], + 0, 0); if (err) goto err; @@ -334,7 +381,8 @@ static int mlx4_init_icm(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap, init_hca->qpc_base, dev_cap->qpc_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0, 0); + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], + 0, 0); if (err) { mlx4_err(dev, "Failed to map QP context memory, aborting.\n"); goto err_unmap_dmpt; @@ -344,7 +392,8 @@ static int mlx4_init_icm(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap, init_hca->auxc_base, dev_cap->aux_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0, 0); + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], + 0, 0); if (err) { mlx4_err(dev, "Failed to map AUXC context memory, aborting.\n"); goto err_unmap_qp; @@ -354,7 +403,8 @@ static int mlx4_init_icm(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap, init_hca->altc_base, dev_cap->altc_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0, 0); + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], + 0, 0); if (err) { mlx4_err(dev, "Failed to map ALTC context memory, aborting.\n"); goto err_unmap_auxc; @@ -364,7 +414,8 @@ static int mlx4_init_icm(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap, init_hca->rdmarc_base, dev_cap->rdmarc_entry_sz << priv->qp_table.rdmarc_shift, dev->caps.num_qps, - dev->caps.reserved_qps, 0, 0); + dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], + 0, 0); if (err) { mlx4_err(dev, "Failed to map RDMARC context memory, aborting\n"); goto err_unmap_altc; diff --git a/drivers/net/mlx4/mcg.c b/drivers/net/mlx4/mcg.c index 57f7f1f..149129a 100644 --- a/drivers/net/mlx4/mcg.c +++ b/drivers/net/mlx4/mcg.c @@ -358,8 +358,8 @@ int mlx4_init_mcg_table(struct mlx4_dev *dev) struct mlx4_priv *priv = mlx4_priv(dev); int err; - err = mlx4_bitmap_init(&priv->mcg_table.bitmap, - dev->caps.num_amgms, dev->caps.num_amgms - 1, 0); + err = mlx4_bitmap_init(&priv->mcg_table.bitmap, dev->caps.num_amgms, + dev->caps.num_amgms - 1, 0, 0); if (err) return err; diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 89d4ccc..a9b3ead 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -111,6 +111,7 @@ struct mlx4_bitmap { u32 last; u32 top; u32 max; + u32 reserved_top; u32 mask; spinlock_t lock; unsigned long *table; @@ -289,7 +290,8 @@ u32 mlx4_bitmap_alloc(struct mlx4_bitmap *bitmap); void mlx4_bitmap_free(struct mlx4_bitmap *bitmap, u32 obj); u32 mlx4_bitmap_alloc_range(struct mlx4_bitmap *bitmap, int cnt, int align); void mlx4_bitmap_free_range(struct mlx4_bitmap *bitmap, u32 obj, int cnt); -int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask, u32 reserved); +int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask, + u32 reserved_bot, u32 resetrved_top); void mlx4_bitmap_cleanup(struct mlx4_bitmap *bitmap); int mlx4_reset(struct mlx4_dev *dev); diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 03a9abc..1df2b36 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -436,7 +436,7 @@ int mlx4_init_mr_table(struct mlx4_dev *dev) int err; err = mlx4_bitmap_init(&mr_table->mpt_bitmap, dev->caps.num_mpts, - ~0, dev->caps.reserved_mrws); + ~0, dev->caps.reserved_mrws, 0); if (err) return err; diff --git a/drivers/net/mlx4/pd.c b/drivers/net/mlx4/pd.c index 3a93c5f..573828a 100644 --- a/drivers/net/mlx4/pd.c +++ b/drivers/net/mlx4/pd.c @@ -62,7 +62,7 @@ int mlx4_init_pd_table(struct mlx4_dev *dev) struct mlx4_priv *priv = mlx4_priv(dev); return mlx4_bitmap_init(&priv->pd_bitmap, dev->caps.num_pds, - (1 << 24) - 1, dev->caps.reserved_pds); + (1 << 24) - 1, dev->caps.reserved_pds, 0); } void mlx4_cleanup_pd_table(struct mlx4_dev *dev) @@ -93,7 +93,7 @@ int mlx4_init_uar_table(struct mlx4_dev *dev) { return mlx4_bitmap_init(&mlx4_priv(dev)->uar_table.bitmap, dev->caps.num_uars, dev->caps.num_uars - 1, - max(128, dev->caps.reserved_uars)); + max(128, dev->caps.reserved_uars), 0); } void mlx4_cleanup_uar_table(struct mlx4_dev *dev) diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c index f4a6dc5..a4397a8 100644 --- a/drivers/net/mlx4/qp.c +++ b/drivers/net/mlx4/qp.c @@ -273,6 +273,7 @@ int mlx4_init_qp_table(struct mlx4_dev *dev) { struct mlx4_qp_table *qp_table = &mlx4_priv(dev)->qp_table; int err; + int reserved_from_top = 0; spin_lock_init(&qp_table->lock); INIT_RADIX_TREE(&dev->qp_table_tree, GFP_ATOMIC); @@ -282,9 +283,40 @@ int mlx4_init_qp_table(struct mlx4_dev *dev) * block of special QPs must be aligned to a multiple of 8, so * round up. */ - dev->caps.sqp_start = ALIGN(dev->caps.reserved_qps, 8); + dev->caps.sqp_start = + ALIGN(dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], 8); + + { + int sort[MLX4_QP_REGION_COUNT]; + int i, j, tmp; + int last_base = dev->caps.num_qps; + + for (i = 1; i < MLX4_QP_REGION_COUNT; ++i) + sort[i] = i; + + for (i = MLX4_QP_REGION_COUNT; i > 0; --i) { + for (j = 2; j < i; ++j) { + if (dev->caps.reserved_qps_cnt[sort[j]] > + dev->caps.reserved_qps_cnt[sort[j - 1]]) { + tmp = sort[j]; + sort[j] = sort[j - 1]; + sort[j - 1] = tmp; + } + } + } + + for (i = 1; i < MLX4_QP_REGION_COUNT; ++i) { + last_base -= dev->caps.reserved_qps_cnt[sort[i]]; + dev->caps.reserved_qps_base[sort[i]] = last_base; + reserved_from_top += + dev->caps.reserved_qps_cnt[sort[i]]; + } + + } + err = mlx4_bitmap_init(&qp_table->bitmap, dev->caps.num_qps, - (1 << 24) - 1, dev->caps.sqp_start + 8); + (1 << 23) - 1, dev->caps.sqp_start + 8, + reserved_from_top); if (err) return err; @@ -297,6 +329,20 @@ void mlx4_cleanup_qp_table(struct mlx4_dev *dev) mlx4_bitmap_cleanup(&mlx4_priv(dev)->qp_table.bitmap); } +int mlx4_qp_get_region(struct mlx4_dev *dev, + enum qp_region region, + int *base_qpn, int *cnt) +{ + if ((region < 0) || (region >= MLX4_QP_REGION_COUNT)) + return -EINVAL; + + *base_qpn = dev->caps.reserved_qps_base[region]; + *cnt = dev->caps.reserved_qps_cnt[region]; + + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_qp_get_region); + int mlx4_qp_query(struct mlx4_dev *dev, struct mlx4_qp *qp, struct mlx4_qp_context *context) { diff --git a/drivers/net/mlx4/srq.c b/drivers/net/mlx4/srq.c index d23f46d..2b50ac8 100644 --- a/drivers/net/mlx4/srq.c +++ b/drivers/net/mlx4/srq.c @@ -244,7 +244,7 @@ int mlx4_init_srq_table(struct mlx4_dev *dev) INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); err = mlx4_bitmap_init(&srq_table->bitmap, dev->caps.num_srqs, - dev->caps.num_srqs - 1, dev->caps.reserved_srqs); + dev->caps.num_srqs - 1, dev->caps.reserved_srqs, 0); if (err) return err; diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 7136b16..2935a21 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -133,6 +133,18 @@ enum { MLX4_STAT_RATE_OFFSET = 5 }; +enum qp_region { + MLX4_QP_REGION_FW = 0, + MLX4_QP_REGION_ETH_ADDR, + MLX4_QP_REGION_FC_ADDR, + MLX4_QP_REGION_FC_EXCH, + MLX4_QP_REGION_COUNT +}; + +enum { + MLX4_NUM_FEXCH = 64 * 1024, +}; + static inline u64 mlx4_fw_ver(u64 major, u64 minor, u64 subminor) { return (major << 32) | (minor << 16) | subminor; @@ -157,7 +169,6 @@ struct mlx4_caps { int max_rq_desc_sz; int max_qp_init_rdma; int max_qp_dest_rdma; - int reserved_qps; int sqp_start; int num_srqs; int max_srq_wqes; @@ -187,6 +198,12 @@ struct mlx4_caps { u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; int max_gso_sz; + int reserved_qps_cnt[MLX4_QP_REGION_COUNT]; + int reserved_qps; + int reserved_qps_base[MLX4_QP_REGION_COUNT]; + int log_num_macs; + int log_num_vlans; + int log_num_prios; }; struct mlx4_buf_list { diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 7f128b2..64cbd19 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -307,4 +307,8 @@ static inline struct mlx4_qp *__mlx4_qp_lookup(struct mlx4_dev *dev, u32 qpn) void mlx4_qp_remove(struct mlx4_dev *dev, struct mlx4_qp *qp); +int mlx4_qp_get_region(struct mlx4_dev *dev, + enum qp_region region, + int *base_qpn, int *cnt); + #endif /* MLX4_QP_H */ -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:32 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:32 +0300 Subject: [ofa-general][PATCH 7/11 v3] mlx4: Multiple completion vectors support Message-ID: <4874BEA4.1090201@mellanox.co.il> mlx4: Multiple completion vectors support The driver now creates a completion EQ for every cpu. While allocating CQ a ULP asks a completion vector number it wants the CQ to be attached to. The number of completion vectors is populated via ib_device.num_comp_vectors Signed-off-by: Yevgeny Petrilin --- drivers/infiniband/hw/mlx4/cq.c | 2 +- drivers/infiniband/hw/mlx4/main.c | 2 +- drivers/net/mlx4/cq.c | 14 ++++++++-- drivers/net/mlx4/eq.c | 47 ++++++++++++++++++++++++------------ drivers/net/mlx4/main.c | 14 ++++++---- drivers/net/mlx4/mlx4.h | 4 +- include/linux/mlx4/device.h | 4 ++- 7 files changed, 57 insertions(+), 30 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 299f208..dbfa1a3 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -221,7 +221,7 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector } err = mlx4_cq_alloc(dev->dev, entries, &cq->buf.mtt, uar, - cq->db.dma, &cq->mcq, 0); + cq->db.dma, &cq->mcq, vector, 0); if (err) goto err_dbmap; diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 098dcd2..7ffcb00 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -567,7 +567,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) mlx4_foreach_port(i, ibdev->ports_map) ibdev->num_ports++; ibdev->ib_dev.phys_port_cnt = ibdev->num_ports; - ibdev->ib_dev.num_comp_vectors = 1; + ibdev->ib_dev.num_comp_vectors = dev->caps.num_comp_vectors; ibdev->ib_dev.dma_device = &dev->pdev->dev; ibdev->ib_dev.uverbs_abi_ver = MLX4_IB_UVERBS_ABI_VERSION; diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 471e54a..18bab87 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -189,7 +189,7 @@ EXPORT_SYMBOL_GPL(mlx4_cq_resize); int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, struct mlx4_uar *uar, u64 db_rec, struct mlx4_cq *cq, - int collapsed) + unsigned vector, int collapsed) { struct mlx4_priv *priv = mlx4_priv(dev); struct mlx4_cq_table *cq_table = &priv->cq_table; @@ -227,7 +227,15 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, cq_context->flags = cpu_to_be32(!!collapsed << 18); cq_context->logsize_usrpage = cpu_to_be32((ilog2(nent) << 24) | uar->index); - cq_context->comp_eqn = priv->eq_table.eq[MLX4_EQ_COMP].eqn; + + if (vector >= dev->caps.num_comp_vectors) { + err = -EINVAL; + goto err_radix; + } + + cq->comp_eq_idx = MLX4_EQ_COMP_CPU0 + vector; + cq_context->comp_eqn = priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + + vector].eqn; cq_context->log_page_size = mtt->page_shift - MLX4_ICM_PAGE_SHIFT; mtt_addr = mlx4_mtt_addr(dev, mtt); @@ -276,7 +284,7 @@ void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq) if (err) mlx4_warn(dev, "HW2SW_CQ failed (%d) for CQN %06x\n", err, cq->cqn); - synchronize_irq(priv->eq_table.eq[MLX4_EQ_COMP].irq); + synchronize_irq(priv->eq_table.eq[cq->comp_eq_idx].irq); spin_lock_irq(&cq_table->lock); radix_tree_delete(&cq_table->tree, cq->cqn); diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c index 35c37a9..8b27746 100644 --- a/drivers/net/mlx4/eq.c +++ b/drivers/net/mlx4/eq.c @@ -265,7 +265,7 @@ static irqreturn_t mlx4_interrupt(int irq, void *dev_ptr) writel(priv->eq_table.clr_mask, priv->eq_table.clr_int); - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < MLX4_EQ_COMP_CPU0 + dev->caps.num_comp_vectors; ++i) work |= mlx4_eq_int(dev, &priv->eq_table.eq[i]); return IRQ_RETVAL(work); @@ -482,7 +482,7 @@ static void mlx4_free_irqs(struct mlx4_dev *dev) if (eq_table->have_irq) free_irq(dev->pdev->irq, dev); - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < MLX4_EQ_COMP_CPU0 + dev->caps.num_comp_vectors; ++i) if (eq_table->eq[i].have_irq) free_irq(eq_table->eq[i].irq, eq_table->eq + i); } @@ -553,6 +553,7 @@ void mlx4_unmap_eq_icm(struct mlx4_dev *dev) int mlx4_init_eq_table(struct mlx4_dev *dev) { struct mlx4_priv *priv = mlx4_priv(dev); + int req_eqs; int err; int i; @@ -573,11 +574,21 @@ int mlx4_init_eq_table(struct mlx4_dev *dev) priv->eq_table.clr_int = priv->clr_base + (priv->eq_table.inta_pin < 32 ? 4 : 0); - err = mlx4_create_eq(dev, dev->caps.num_cqs + MLX4_NUM_SPARE_EQE, - (dev->flags & MLX4_FLAG_MSI_X) ? MLX4_EQ_COMP : 0, - &priv->eq_table.eq[MLX4_EQ_COMP]); - if (err) - goto err_out_unmap; + dev->caps.num_comp_vectors = 0; + req_eqs = (dev->flags & MLX4_FLAG_MSI_X) ? num_online_cpus() : 1; + while (req_eqs) { + err = mlx4_create_eq( + dev, dev->caps.num_cqs + MLX4_NUM_SPARE_EQE, + (dev->flags & MLX4_FLAG_MSI_X) ? + (MLX4_EQ_COMP_CPU0 + dev->caps.num_comp_vectors) : 0, + &priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + + dev->caps.num_comp_vectors]); + if (err) + goto err_out_comp; + + dev->caps.num_comp_vectors++; + req_eqs--; + } err = mlx4_create_eq(dev, MLX4_NUM_ASYNC_EQE + MLX4_NUM_SPARE_EQE, (dev->flags & MLX4_FLAG_MSI_X) ? MLX4_EQ_ASYNC : 0, @@ -586,12 +597,16 @@ int mlx4_init_eq_table(struct mlx4_dev *dev) goto err_out_comp; if (dev->flags & MLX4_FLAG_MSI_X) { - static const char *eq_name[] = { - [MLX4_EQ_COMP] = DRV_NAME " (comp)", - [MLX4_EQ_ASYNC] = DRV_NAME " (async)" - }; + static char eq_name[MLX4_NUM_EQ][20]; + + for (i = 0; i < MLX4_EQ_COMP_CPU0 + + dev->caps.num_comp_vectors; ++i) { + if (i == 0) + snprintf(eq_name[0], 20, DRV_NAME "(async)"); + else + snprintf(eq_name[i], 20, "comp_" DRV_NAME "%d", + i - 1); - for (i = 0; i < MLX4_NUM_EQ; ++i) { err = request_irq(priv->eq_table.eq[i].irq, mlx4_msi_x_interrupt, 0, eq_name[i], priv->eq_table.eq + i); @@ -616,7 +631,7 @@ int mlx4_init_eq_table(struct mlx4_dev *dev) mlx4_warn(dev, "MAP_EQ for async EQ %d failed (%d)\n", priv->eq_table.eq[MLX4_EQ_ASYNC].eqn, err); - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < MLX4_EQ_COMP_CPU0 + dev->caps.num_comp_vectors; ++i) eq_set_ci(&priv->eq_table.eq[i], 1); return 0; @@ -625,9 +640,9 @@ err_out_async: mlx4_free_eq(dev, &priv->eq_table.eq[MLX4_EQ_ASYNC]); err_out_comp: - mlx4_free_eq(dev, &priv->eq_table.eq[MLX4_EQ_COMP]); + for (i = 0; i < dev->caps.num_comp_vectors; ++i) + mlx4_free_eq(dev, &priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + i]); -err_out_unmap: mlx4_unmap_clr_int(dev); mlx4_free_irqs(dev); @@ -646,7 +661,7 @@ void mlx4_cleanup_eq_table(struct mlx4_dev *dev) mlx4_free_irqs(dev); - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < MLX4_EQ_COMP_CPU0 + dev->caps.num_comp_vectors; ++i) mlx4_free_eq(dev, &priv->eq_table.eq[i]); mlx4_unmap_clr_int(dev); diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index bb96445..c078f82 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -898,22 +898,24 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev) { struct mlx4_priv *priv = mlx4_priv(dev); struct msix_entry entries[MLX4_NUM_EQ]; + int needed_vectors = MLX4_EQ_COMP_CPU0 + num_online_cpus(); int err; int i; if (msi_x) { - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < needed_vectors; ++i) entries[i].entry = i; - err = pci_enable_msix(dev->pdev, entries, ARRAY_SIZE(entries)); + err = pci_enable_msix(dev->pdev, entries, needed_vectors); if (err) { if (err > 0) - mlx4_info(dev, "Only %d MSI-X vectors available, " - "not using MSI-X\n", err); + mlx4_info(dev, "Only %d MSI-X vectors " + "available, need %d. Not using MSI-X\n", + err, needed_vectors); goto no_msi; } - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < needed_vectors; ++i) priv->eq_table.eq[i].irq = entries[i].vector; dev->flags |= MLX4_FLAG_MSI_X; @@ -921,7 +923,7 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev) } no_msi: - for (i = 0; i < MLX4_NUM_EQ; ++i) + for (i = 0; i < needed_vectors; ++i) priv->eq_table.eq[i].irq = dev->pdev->irq; } diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index d8278ac..77d00f4 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -64,8 +64,8 @@ enum { enum { MLX4_EQ_ASYNC, - MLX4_EQ_COMP, - MLX4_NUM_EQ + MLX4_EQ_COMP_CPU0, + MLX4_NUM_EQ = MLX4_EQ_COMP_CPU0 + NR_CPUS }; enum { diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index d97314d..7cbe078 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -187,6 +187,7 @@ struct mlx4_caps { int reserved_cqs; int num_eqs; int reserved_eqs; + int num_comp_vectors; int num_mpts; int num_mtt_segs; int fmr_reserved_mtts; @@ -305,6 +306,7 @@ struct mlx4_cq { int arm_sn; int cqn; + int comp_eq_idx; atomic_t refcount; struct completion free; @@ -434,7 +436,7 @@ void mlx4_free_hwq_res(struct mlx4_dev *mdev, struct mlx4_hwq_resources *wqres, int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, struct mlx4_uar *uar, u64 db_rec, struct mlx4_cq *cq, - int collapsed); + unsigned vector, int collapsed); void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq); int mlx4_qp_reserve_range(struct mlx4_dev *dev, int cnt, int align, int *base); -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:16 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:16 +0300 Subject: [ofa-general][PATCH 4/11 v3] mlx4: Port Ethernet mtu capabilities handle Message-ID: <4874BE94.6080205@mellanox.co.il> mlx4: Port Ethernet mtu capabilities handle Ethernet max mtu and default Mac address are revealed through QUERY_DEV_CAP command. Signed-off-by: Yevgeny Petrilin --- drivers/net/mlx4/fw.c | 11 ++++++----- drivers/net/mlx4/fw.h | 4 +++- drivers/net/mlx4/main.c | 4 +++- include/linux/mlx4/device.h | 4 +++- 4 files changed, 15 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index e875b08..1cbc30f 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -314,7 +314,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) MLX4_GET(field, outbox, QUERY_DEV_CAP_VL_PORT_OFFSET); dev_cap->max_vl[i] = field >> 4; MLX4_GET(field, outbox, QUERY_DEV_CAP_MTU_WIDTH_OFFSET); - dev_cap->max_mtu[i] = field >> 4; + dev_cap->ib_mtu[i] = field >> 4; dev_cap->max_port_width[i] = field & 0xf; MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GID_OFFSET); dev_cap->max_gids[i] = 1 << (field & 0xf); @@ -339,7 +339,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) QUERY_PORT_SUPPORTED_TYPE_OFFSET); dev_cap->supported_port_types[i] = field & 3; MLX4_GET(field, outbox, QUERY_PORT_MTU_OFFSET); - dev_cap->max_mtu[i] = field & 0xf; + dev_cap->ib_mtu[i] = field & 0xf; MLX4_GET(field, outbox, QUERY_PORT_WIDTH_OFFSET); dev_cap->max_port_width[i] = field & 0xf; MLX4_GET(field, outbox, QUERY_PORT_MAX_GID_PKEY_OFFSET); @@ -350,7 +350,8 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) MLX4_GET(field, outbox, QUERY_PORT_MAX_MACVLAN_OFFSET); dev_cap->log_max_macs[i] = field & 0xf; dev_cap->log_max_vlans[i] = field >> 4; - + dev_cap->eth_mtu[i] = be16_to_cpu(((u16 *) outbox)[1]); + dev_cap->def_mac[i] = be64_to_cpu(((u64 *) outbox)[2]); } } @@ -388,7 +389,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) mlx4_dbg(dev, "Max CQEs: %d, max WQEs: %d, max SRQ WQEs: %d\n", dev_cap->max_cq_sz, dev_cap->max_qp_sz, dev_cap->max_srq_sz); mlx4_dbg(dev, "Local CA ACK delay: %d, max MTU: %d, port width cap: %d\n", - dev_cap->local_ca_ack_delay, 128 << dev_cap->max_mtu[1], + dev_cap->local_ca_ack_delay, 128 << dev_cap->ib_mtu[1], dev_cap->max_port_width[1]); mlx4_dbg(dev, "Max SQ desc size: %d, max SQ S/G: %d\n", dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); @@ -796,7 +797,7 @@ int mlx4_INIT_PORT(struct mlx4_dev *dev, int port) flags |= (dev->caps.port_width_cap[port] & 0xf) << INIT_PORT_PORT_WIDTH_SHIFT; MLX4_PUT(inbox, flags, INIT_PORT_FLAGS_OFFSET); - field = 128 << dev->caps.mtu_cap[port]; + field = 128 << dev->caps.ib_mtu_cap[port]; MLX4_PUT(inbox, field, INIT_PORT_MTU_OFFSET); field = dev->caps.gid_table_len[port]; MLX4_PUT(inbox, field, INIT_PORT_MAX_GID_OFFSET); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index 50a6a7d..ef964d5 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -61,11 +61,13 @@ struct mlx4_dev_cap { int local_ca_ack_delay; int num_ports; u32 max_msg_sz; - int max_mtu[MLX4_MAX_PORTS + 1]; + int ib_mtu[MLX4_MAX_PORTS + 1]; int max_port_width[MLX4_MAX_PORTS + 1]; int max_vl[MLX4_MAX_PORTS + 1]; int max_gids[MLX4_MAX_PORTS + 1]; int max_pkeys[MLX4_MAX_PORTS + 1]; + u64 def_mac[MLX4_MAX_PORTS + 1]; + int eth_mtu[MLX4_MAX_PORTS + 1]; u16 stat_rate_support; u32 flags; int reserved_uars; diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 0f52e1a..531add4 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -171,10 +171,12 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.num_ports = dev_cap->num_ports; for (i = 1; i <= dev->caps.num_ports; ++i) { dev->caps.vl_cap[i] = dev_cap->max_vl[i]; - dev->caps.mtu_cap[i] = dev_cap->max_mtu[i]; + dev->caps.ib_mtu_cap[i] = dev_cap->ib_mtu[i]; dev->caps.gid_table_len[i] = dev_cap->max_gids[i]; dev->caps.pkey_table_len[i] = dev_cap->max_pkeys[i]; dev->caps.port_width_cap[i] = dev_cap->max_port_width[i]; + dev->caps.eth_mtu_cap[i] = dev_cap->eth_mtu[i]; + dev->caps.def_mac[i] = dev_cap->def_mac[i]; } dev->caps.num_uars = dev_cap->uar_size / PAGE_SIZE; diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index ec60b7b..a076b59 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -160,7 +160,9 @@ struct mlx4_caps { u64 fw_ver; int num_ports; int vl_cap[MLX4_MAX_PORTS + 1]; - int mtu_cap[MLX4_MAX_PORTS + 1]; + int ib_mtu_cap[MLX4_MAX_PORTS + 1]; + u64 def_mac[MLX4_MAX_PORTS + 1]; + int eth_mtu_cap[MLX4_MAX_PORTS + 1]; int gid_table_len[MLX4_MAX_PORTS + 1]; int pkey_table_len[MLX4_MAX_PORTS + 1]; int local_ca_ack_delay; -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:37 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:37 +0300 Subject: [ofa-general][PATCH 8/11 v3] mlx4: Default value for automatic completion vector selection Message-ID: <4874BEA9.4010202@mellanox.co.il> mlx4: Default value for automatic completion vector selection When the vector number passed to mlx4_cq_alloc is IB_CQ_VECTOR_LEAST_ATTACHED (0xff), the driver selects the completion vector that has the least CQ's attached to it and attaches the CQ to the chosen vector. IB_CQ_VECTOR_LEAST_ATTACHED is redefined in device.h as MLX4_ANY_VECTOR because we don't want all mlx4_core clients (Ethernet and FCoE) to include Signed-off-by: Yevgeny Petrilin --- drivers/net/mlx4/cq.c | 22 +++++++++++++++++++++- drivers/net/mlx4/mlx4.h | 1 + include/linux/mlx4/device.h | 4 ++++ include/rdma/ib_verbs.h | 10 +++++++++- 4 files changed, 35 insertions(+), 2 deletions(-) diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 18bab87..89fa505 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -187,6 +187,22 @@ int mlx4_cq_resize(struct mlx4_dev *dev, struct mlx4_cq *cq, } EXPORT_SYMBOL_GPL(mlx4_cq_resize); +static int mlx4_find_least_loaded_vector(struct mlx4_priv *priv) +{ + int i; + int index = 0; + int min = priv->eq_table.eq[MLX4_EQ_COMP_CPU0].load; + + for (i = 1; i < priv->dev.caps.num_comp_vectors; i++) { + if (priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + i].load < min) { + index = i; + min = priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + i].load; + } + } + + return index; +} + int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, struct mlx4_uar *uar, u64 db_rec, struct mlx4_cq *cq, unsigned vector, int collapsed) @@ -228,7 +244,9 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, cq_context->flags = cpu_to_be32(!!collapsed << 18); cq_context->logsize_usrpage = cpu_to_be32((ilog2(nent) << 24) | uar->index); - if (vector >= dev->caps.num_comp_vectors) { + if (vector == MLX4_ANY_VECTOR) + vector = mlx4_find_least_loaded_vector(priv); + else if (vector >= dev->caps.num_comp_vectors) { err = -EINVAL; goto err_radix; } @@ -248,6 +266,7 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, if (err) goto err_radix; + priv->eq_table.eq[cq->comp_eq_idx].load++; cq->cons_index = 0; cq->arm_sn = 1; cq->uar = uar; @@ -285,6 +304,7 @@ void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq) mlx4_warn(dev, "HW2SW_CQ failed (%d) for CQN %06x\n", err, cq->cqn); synchronize_irq(priv->eq_table.eq[cq->comp_eq_idx].irq); + priv->eq_table.eq[cq->comp_eq_idx].load--; spin_lock_irq(&cq_table->lock); radix_tree_delete(&cq_table->tree, cq->cqn); diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 77d00f4..8a7bae5 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -144,6 +144,7 @@ struct mlx4_eq { u16 irq; u16 have_irq; int nent; + int load; struct mlx4_buf_list *page_list; struct mlx4_mtt mtt; }; diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 7cbe078..ae70add 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -37,6 +37,8 @@ #include #include +#include + #include enum { @@ -151,6 +153,8 @@ enum { MLX4_NUM_FEXCH = 64 * 1024, }; +#define MLX4_ANY_VECTOR IB_CQ_VECTOR_LEAST_ATTACHED + static inline u64 mlx4_fw_ver(u64 major, u64 minor, u64 subminor) { return (major << 32) | (minor << 16) | subminor; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 4c3c76b..34df9a7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1393,6 +1393,13 @@ static inline int ib_post_recv(struct ib_qp *qp, return qp->device->post_recv(qp, recv_wr, bad_recv_wr); } +/* + * IB_CQ_VECTOR_LEAST_ATTACHED: The constatnt specifies that + * teh cq will be attached to the least attached + * completion vector + */ +#define IB_CQ_VECTOR_LEAST_ATTACHED 0xff + /** * ib_create_cq - Creates a CQ on the specified device. * @device: The device on which to create the CQ. @@ -1404,7 +1411,8 @@ static inline int ib_post_recv(struct ib_qp *qp, * the associated completion and event handlers. * @cqe: The minimum size of the CQ. * @comp_vector - Completion vector used to signal completion events. - * Must be >= 0 and < context->num_comp_vectors. + * Must be >= 0 and < context->num_comp_vectors + * or IB_CQ_VECTOR_LEAST_ATTACHED. * * Users can examine the cq structure to determine the actual CQ size. */ -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:41 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:41 +0300 Subject: [ofa-general][PATCH 9/11 v3] mlx4: Fibre Channel support Message-ID: <4874BEAD.9020700@mellanox.co.il> mlx4: Fibre Channel support As we did with QPs, some of the MPTs are pre-reserved (the MPTs that are mapped for FEXCHs, 2*64K of them). So needed to split the operation of allocating an MPT to two: The allocation of a bit from the bitmap The actual creation of the entry (and it's MTT). So, mr_alloc_reserved() is the second part, where you know which MPT number was allocated. mr_alloc() is the one that allocates a number from the bitmap. Normal users keep using the original mr_alloc(). For FEXCH, when we know the pre-reserved MPT entry, we call mr_alloc_reserved() directly. Same with the mr_free() and corresponding mr_free_reserved(). The first will just put back the bit, the later will actually destroy the entry, but will leave the bit set. map_phys_fmr_fbo() is very much like the original map_phys_fmr() allows setting an FBO (First Byte Offset) for the MPT allows setting the data length for the MPT does not increase the higher bits of the key after every map. Signed-off-by: Yevgeny Petrilin --- drivers/infiniband/hw/mlx4/main.c | 4 +- drivers/net/mlx4/main.c | 2 +- drivers/net/mlx4/mr.c | 128 ++++++++++++++++++++++++++++++++----- include/linux/mlx4/device.h | 19 ++++++ include/linux/mlx4/qp.h | 11 +++- 5 files changed, 145 insertions(+), 19 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 7ffcb00..efef0d1 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -118,7 +118,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, dev->dev->caps.max_rq_sg); props->max_cq = dev->dev->caps.num_cqs - dev->dev->caps.reserved_cqs; props->max_cqe = dev->dev->caps.max_cqes; - props->max_mr = dev->dev->caps.num_mpts - dev->dev->caps.reserved_mrws; + props->max_mr = dev->dev->caps.num_mpts - + dev->dev->caps.reserved_mrws - + dev->dev->caps.num_fexch_mpts; props->max_pd = dev->dev->caps.num_pds - dev->dev->caps.reserved_pds; props->max_qp_rd_atom = dev->dev->caps.max_qp_dest_rdma; props->max_qp_init_rd_atom = dev->dev->caps.max_qp_init_rdma; diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index c078f82..12600a9 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -81,7 +81,7 @@ static struct mlx4_profile default_profile = { .rdmarc_per_qp = 1 << 4, .num_cq = 1 << 16, .num_mcg = 1 << 13, - .num_mpt = 1 << 17, + .num_mpt = 1 << 18, .num_mtt = 1 << 20, }; diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 1df2b36..1a5d23c 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -52,7 +52,9 @@ struct mlx4_mpt_entry { __be64 length; __be32 lkey; __be32 win_cnt; - u8 reserved1[3]; + u8 reserved1; + u8 flags2; + u8 reserved2; u8 mtt_rep; __be64 mtt_seg; __be32 mtt_sz; @@ -68,6 +70,8 @@ struct mlx4_mpt_entry { #define MLX4_MTT_FLAG_PRESENT 1 +#define MLX4_MPT_FLAG2_FBO_EN (1 << 7) + #define MLX4_MPT_STATUS_SW 0xF0 #define MLX4_MPT_STATUS_HW 0x00 @@ -250,6 +254,21 @@ static int mlx4_HW2SW_MPT(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox !mailbox, MLX4_CMD_HW2SW_MPT, MLX4_CMD_TIME_CLASS_B); } +int mlx4_mr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, + u64 iova, u64 size, u32 access, int npages, + int page_shift, struct mlx4_mr *mr) +{ + mr->iova = iova; + mr->size = size; + mr->pd = pd; + mr->access = access; + mr->enabled = 0; + mr->key = hw_index_to_key(mridx); + + return mlx4_mtt_init(dev, npages, page_shift, &mr->mtt); +} +EXPORT_SYMBOL_GPL(mlx4_mr_alloc_reserved); + int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, int npages, int page_shift, struct mlx4_mr *mr) { @@ -261,14 +280,8 @@ int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, if (index == -1) return -ENOMEM; - mr->iova = iova; - mr->size = size; - mr->pd = pd; - mr->access = access; - mr->enabled = 0; - mr->key = hw_index_to_key(index); - - err = mlx4_mtt_init(dev, npages, page_shift, &mr->mtt); + err = mlx4_mr_alloc_reserved(dev, index, pd, iova, size, + access, npages, page_shift, mr); if (err) mlx4_bitmap_free(&priv->mr_table.mpt_bitmap, index); @@ -276,9 +289,8 @@ int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, } EXPORT_SYMBOL_GPL(mlx4_mr_alloc); -void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr) +void mlx4_mr_free_reserved(struct mlx4_dev *dev, struct mlx4_mr *mr) { - struct mlx4_priv *priv = mlx4_priv(dev); int err; if (mr->enabled) { @@ -290,6 +302,13 @@ void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr) } mlx4_mtt_cleanup(dev, &mr->mtt); +} +EXPORT_SYMBOL_GPL(mlx4_mr_free_reserved); + +void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + mlx4_mr_free_reserved(dev, mr); mlx4_bitmap_free(&priv->mr_table.mpt_bitmap, key_to_hw_index(mr->key)); } EXPORT_SYMBOL_GPL(mlx4_mr_free); @@ -435,8 +454,16 @@ int mlx4_init_mr_table(struct mlx4_dev *dev) struct mlx4_mr_table *mr_table = &mlx4_priv(dev)->mr_table; int err; + if (!is_power_of_2(dev->caps.num_mpts)) + return -EINVAL; + + dev->caps.num_fexch_mpts = + 2 * dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FC_EXCH]; + dev->caps.reserved_fexch_mpts_base = dev->caps.num_mpts - + dev->caps.num_fexch_mpts; err = mlx4_bitmap_init(&mr_table->mpt_bitmap, dev->caps.num_mpts, - ~0, dev->caps.reserved_mrws, 0); + ~0, dev->caps.reserved_mrws, + dev->caps.reserved_fexch_mpts_base); if (err) return err; @@ -500,8 +527,9 @@ static inline int mlx4_check_fmr(struct mlx4_fmr *fmr, u64 *page_list, return 0; } -int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, - int npages, u64 iova, u32 *lkey, u32 *rkey) +int mlx4_map_phys_fmr_fbo(struct mlx4_dev *dev, struct mlx4_fmr *fmr, + u64 *page_list, int npages, u64 iova, u32 fbo, + u32 len, u32 *lkey, u32 *rkey, int same_key) { u32 key; int i, err; @@ -513,7 +541,8 @@ int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list ++fmr->maps; key = key_to_hw_index(fmr->mr.key); - key += dev->caps.num_mpts; + if (!same_key) + key += dev->caps.num_mpts; *lkey = *rkey = fmr->mr.key = hw_index_to_key(key); *(u8 *) fmr->mpt = MLX4_MPT_STATUS_SW; @@ -529,8 +558,10 @@ int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list fmr->mpt->key = cpu_to_be32(key); fmr->mpt->lkey = cpu_to_be32(key); - fmr->mpt->length = cpu_to_be64(npages * (1ull << fmr->page_shift)); + fmr->mpt->length = cpu_to_be64(len); fmr->mpt->start = cpu_to_be64(iova); + fmr->mpt->first_byte_offset = cpu_to_be32(fbo & 0x001fffff); + fmr->mpt->flags2 = (fbo ? MLX4_MPT_FLAG2_FBO_EN : 0); /* Make MTT entries are visible before setting MPT status */ wmb(); @@ -542,6 +573,16 @@ int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list return 0; } +EXPORT_SYMBOL_GPL(mlx4_map_phys_fmr_fbo); + +int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, + int npages, u64 iova, u32 *lkey, u32 *rkey) +{ + u32 len = npages * (1ull << fmr->page_shift); + + return mlx4_map_phys_fmr_fbo(dev, fmr, page_list, npages, iova, 0, + len, lkey, rkey, 1); +} EXPORT_SYMBOL_GPL(mlx4_map_phys_fmr); int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, @@ -586,6 +627,49 @@ err_free: } EXPORT_SYMBOL_GPL(mlx4_fmr_alloc); +int mlx4_fmr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, + u32 pd, u32 access, int max_pages, + int max_maps, u8 page_shift, struct mlx4_fmr *fmr) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + u64 mtt_seg; + int err = -ENOMEM; + + if (page_shift < (ffs(dev->caps.page_size_cap) - 1) || page_shift >= 32) + return -EINVAL; + + /* All MTTs must fit in the same page */ + if (max_pages * sizeof *fmr->mtts > PAGE_SIZE) + return -EINVAL; + + fmr->page_shift = page_shift; + fmr->max_pages = max_pages; + fmr->max_maps = max_maps; + fmr->maps = 0; + + err = mlx4_mr_alloc_reserved(dev, mridx, pd, 0, 0, access, max_pages, + page_shift, &fmr->mr); + if (err) + return err; + + mtt_seg = fmr->mr.mtt.first_seg * dev->caps.mtt_entry_sz; + + fmr->mtts = mlx4_table_find(&priv->mr_table.mtt_table, + fmr->mr.mtt.first_seg, + &fmr->dma_handle); + if (!fmr->mtts) { + err = -ENOMEM; + goto err_free; + } + + return 0; + +err_free: + mlx4_mr_free_reserved(dev, &fmr->mr); + return err; +} +EXPORT_SYMBOL_GPL(mlx4_fmr_alloc_reserved); + int mlx4_fmr_enable(struct mlx4_dev *dev, struct mlx4_fmr *fmr) { struct mlx4_priv *priv = mlx4_priv(dev); @@ -628,6 +712,18 @@ int mlx4_fmr_free(struct mlx4_dev *dev, struct mlx4_fmr *fmr) } EXPORT_SYMBOL_GPL(mlx4_fmr_free); +int mlx4_fmr_free_reserved(struct mlx4_dev *dev, struct mlx4_fmr *fmr) +{ + if (fmr->maps) + return -EBUSY; + + fmr->mr.enabled = 0; + mlx4_mr_free_reserved(dev, &fmr->mr); + + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_fmr_free_reserved); + int mlx4_SYNC_TPT(struct mlx4_dev *dev) { return mlx4_cmd(dev, 0, 0, 0, MLX4_CMD_SYNC_TPT, 1000); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index ae70add..d1ecfdc 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -218,6 +218,8 @@ struct mlx4_caps { int log_num_vlans; int log_num_prios; enum mlx4_port_type port_type[MLX4_MAX_PORTS + 1]; + int reserved_fexch_mpts_base; + int num_fexch_mpts; }; struct mlx4_buf_list { @@ -398,6 +400,12 @@ static inline u32 mlx4_get_ports_of_type(struct mlx4_dev *dev, for ((port) = 1; (port) <= MLX4_MAX_PORTS; (port)++) \ if (bitmap & 1 << ((port)-1)) + +static inline int mlx4_get_fexch_mpts_base(struct mlx4_dev *dev) +{ + return dev->caps.reserved_fexch_mpts_base; +} + int mlx4_buf_alloc(struct mlx4_dev *dev, int size, int max_direct, struct mlx4_buf *buf); void mlx4_buf_free(struct mlx4_dev *dev, int size, struct mlx4_buf *buf); @@ -421,8 +429,12 @@ int mlx4_mtt_init(struct mlx4_dev *dev, int npages, int page_shift, void mlx4_mtt_cleanup(struct mlx4_dev *dev, struct mlx4_mtt *mtt); u64 mlx4_mtt_addr(struct mlx4_dev *dev, struct mlx4_mtt *mtt); +int mlx4_mr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, + u64 iova, u64 size, u32 access, int npages, + int page_shift, struct mlx4_mr *mr); int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, int npages, int page_shift, struct mlx4_mr *mr); +void mlx4_mr_free_reserved(struct mlx4_dev *dev, struct mlx4_mr *mr); void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr); int mlx4_mr_enable(struct mlx4_dev *dev, struct mlx4_mr *mr); int mlx4_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt, @@ -467,13 +479,20 @@ void mlx4_unregister_mac(struct mlx4_dev *dev, u8 port, int index); int mlx4_register_vlan(struct mlx4_dev *dev, u8 port, u16 vlan, int *index); void mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, int index); +int mlx4_map_phys_fmr_fbo(struct mlx4_dev *dev, struct mlx4_fmr *fmr, + u64 *page_list, int npages, u64 iova, u32 fbo, + u32 len, u32 *lkey, u32 *rkey, int same_key); int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, int npages, u64 iova, u32 *lkey, u32 *rkey); +int mlx4_fmr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, + u32 access, int max_pages, int max_maps, + u8 page_shift, struct mlx4_fmr *fmr); int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, int max_maps, u8 page_shift, struct mlx4_fmr *fmr); int mlx4_fmr_enable(struct mlx4_dev *dev, struct mlx4_fmr *fmr); void mlx4_fmr_unmap(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u32 *lkey, u32 *rkey); +int mlx4_fmr_free_reserved(struct mlx4_dev *dev, struct mlx4_fmr *fmr); int mlx4_fmr_free(struct mlx4_dev *dev, struct mlx4_fmr *fmr); int mlx4_SYNC_TPT(struct mlx4_dev *dev); diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index f675ded..35689e0 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -151,7 +151,16 @@ struct mlx4_qp_context { u8 reserved4[2]; u8 mtt_base_addr_h; __be32 mtt_base_addr_l; - u32 reserved5[10]; + u8 VE; + u8 reserved5; + __be16 VFT_id_prio; + u8 reserved6; + u8 exch_size; + __be16 exch_base; + u8 VFT_hop_cnt; + u8 my_fc_id_idx; + __be16 reserved7; + u32 reserved8[7]; }; /* Which firmware version adds support for NEC (NoErrorCompletion) bit */ -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:35:45 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:35:45 +0300 Subject: [ofa-general][PATCH 10/11 v3] mlx4_core: Auto negotiation support Message-ID: <4874BEB1.8070908@mellanox.co.il> mlx4_core: Auto negotiation support At any time when port link is down (except to driver restart), and port is configured to auto sensing, we try to sense port to configuration in order to determine how to initialize the port. If port type need to be changed, all ulp's are unregistered and then registered again with the new port types. Sense is done with intervals that move between 1-3 seconds. Signed-off-by: Yevgeny Petrilin --- drivers/net/mlx4/Makefile | 2 +- drivers/net/mlx4/eq.c | 16 +++-- drivers/net/mlx4/intf.c | 4 + drivers/net/mlx4/main.c | 97 +++++++++++++++++++++++----- drivers/net/mlx4/mlx4.h | 24 +++++++ drivers/net/mlx4/sense.c | 148 +++++++++++++++++++++++++++++++++++++++++++ include/linux/mlx4/cmd.h | 1 + include/linux/mlx4/device.h | 6 +- 8 files changed, 272 insertions(+), 26 deletions(-) create mode 100644 drivers/net/mlx4/sense.c diff --git a/drivers/net/mlx4/Makefile b/drivers/net/mlx4/Makefile index f4932d8..3f71687 100644 --- a/drivers/net/mlx4/Makefile +++ b/drivers/net/mlx4/Makefile @@ -1,4 +1,4 @@ obj-$(CONFIG_MLX4_CORE) += mlx4_core.o mlx4_core-y := alloc.o catas.o cmd.o cq.o eq.o fw.o icm.o intf.o main.o mcg.o \ - mr.o pd.o profile.o qp.o reset.o srq.o port.o + mr.o pd.o profile.o qp.o reset.o srq.o port.o sense.o diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c index 8b27746..9615f53 100644 --- a/drivers/net/mlx4/eq.c +++ b/drivers/net/mlx4/eq.c @@ -162,6 +162,7 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq) int cqn; int eqes_found = 0; int set_ci = 0; + int port; while ((eqe = next_eqe_sw(eq))) { /* @@ -202,11 +203,16 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq) break; case MLX4_EVENT_TYPE_PORT_CHANGE: - mlx4_dispatch_event(dev, - eqe->subtype == MLX4_PORT_CHANGE_SUBTYPE_ACTIVE ? - MLX4_DEV_EVENT_PORT_UP : - MLX4_DEV_EVENT_PORT_DOWN, - be32_to_cpu(eqe->event.port_change.port) >> 28); + port = be32_to_cpu(eqe->event.port_change.port) >> 28; + if (eqe->subtype == MLX4_PORT_CHANGE_SUBTYPE_DOWN) { + mlx4_dispatch_event(dev, MLX4_DEV_EVENT_PORT_DOWN, + port); + mlx4_priv(dev)->sense.do_sense_port[port] = 1; + } else { + mlx4_dispatch_event(dev, MLX4_DEV_EVENT_PORT_UP, + port); + mlx4_priv(dev)->sense.do_sense_port[port] = 0; + } break; case MLX4_EVENT_TYPE_CQ_ERROR: diff --git a/drivers/net/mlx4/intf.c b/drivers/net/mlx4/intf.c index 4a6c4d5..75fdc4f 100644 --- a/drivers/net/mlx4/intf.c +++ b/drivers/net/mlx4/intf.c @@ -140,6 +140,8 @@ int mlx4_register_device(struct mlx4_dev *dev) mutex_unlock(&intf_mutex); mlx4_start_catas_poll(dev); + mlx4_start_sense(dev); + return 0; } @@ -148,6 +150,8 @@ void mlx4_unregister_device(struct mlx4_dev *dev) struct mlx4_priv *priv = mlx4_priv(dev); struct mlx4_interface *intf; + mlx4_stop_sense(dev); + mlx4_stop_catas_poll(dev); mutex_lock(&intf_mutex); diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index aae3460..a812fe2 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -103,18 +103,21 @@ module_param_array_named(port_type, port_type_arr, charp, NULL, 0444); MODULE_PARM_DESC(port_type, "Ports L2 type (ib/eth/auto, entry per port, " "comma seperated, default ib for all)"); -static int mlx4_check_port_params(struct mlx4_dev *dev, - enum mlx4_port_type *port_type) +int mlx4_check_port_params(struct mlx4_dev *dev, + enum mlx4_port_type *port_type) { - if (port_type[0] != port_type[1] && + if ((port_type[0] != port_type[1] || + port_type[0] == MLX4_PORT_TYPE_AUTO) && !(dev->caps.flags & MLX4_DEV_CAP_FLAG_DPDP)) { mlx4_err(dev, "Only same port types supported " "on this HCA, aborting.\n"); return -EINVAL; } - if ((port_type[0] == MLX4_PORT_TYPE_ETH) && - (port_type[1] == MLX4_PORT_TYPE_IB)) { - mlx4_err(dev, "eth-ib configuration is not supported.\n"); + if ((port_type[0] == MLX4_PORT_TYPE_ETH && + port_type[1] != MLX4_PORT_TYPE_ETH) || + (port_type[1] == MLX4_PORT_TYPE_IB && + port_type[0] != MLX4_PORT_TYPE_IB)) { + mlx4_err(dev, "Given port configuration is not supported.\n"); return -EINVAL; } return 0; @@ -128,8 +131,10 @@ static void mlx4_str2port_type(char **port_str, for (i = 0; i < MLX4_MAX_PORTS; i++) { if (!strcmp(port_str[i], "eth")) port_type[i] = MLX4_PORT_TYPE_ETH; - else + else if (!strcmp(port_str[i], "ib")) port_type[i] = MLX4_PORT_TYPE_IB; + else + port_type[i] = MLX4_PORT_TYPE_AUTO; } } @@ -226,11 +231,17 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) mlx4_warn(dev, "FW doesn't support Multi Protocol, " "loading IB only\n"); dev->caps.port_type[i] = MLX4_PORT_TYPE_IB; + dev->caps.possible_type[i] = MLX4_PORT_TYPE_IB; continue; } - if (port_type[i-1] & dev_cap->supported_port_types[i]) - dev->caps.port_type[i] = port_type[i-1]; - else { + mlx4_priv(dev)->sense.sense_allowed[i] = + (dev_cap->supported_port_types[i] >> 1) & 1; + if (port_type[i-1] & dev_cap->supported_port_types[i]) { + dev->caps.possible_type[i] = port_type[i-1]; + dev->caps.port_type[i] = + (port_type[i-1] == MLX4_PORT_TYPE_ETH) ? + port_type[i-1] : MLX4_PORT_TYPE_IB; + } else { mlx4_err(dev, "Requested port type for port %d " "not supported by HW\n", i); return -ENODEV; @@ -268,7 +279,7 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) /* Changes the port configuration of the device. * Every user of this function must hold the port lock */ -static int mlx4_change_port_types(struct mlx4_dev *dev, +int mlx4_change_port_types(struct mlx4_dev *dev, enum mlx4_port_type *port_types) { int err = 0; @@ -279,10 +290,13 @@ static int mlx4_change_port_types(struct mlx4_dev *dev, if (port_types[port] != dev->caps.port_type[port + 1]) { change = 1; dev->caps.port_type[port + 1] = port_types[port]; + if (dev->caps.possible_type[port + 1] != MLX4_PORT_TYPE_AUTO) + dev->caps.possible_type[port + 1] = port_types[port]; } } if (change) { mlx4_unregister_device(dev); + flush_workqueue(mlx4_priv(dev)->sense.sense_wq); for (port = 1; port <= dev->caps.num_ports; port++) { mlx4_CLOSE_PORT(dev, port); err = mlx4_SET_PORT(dev, port); @@ -306,9 +320,15 @@ static ssize_t show_port_type(struct device *dev, struct mlx4_port_info *info = container_of(attr, struct mlx4_port_info, port_attr); struct mlx4_dev *mdev = info->dev; + char type[8]; - sprintf(buf, "%s\n", (mdev->caps.port_type[info->port] == MLX4_PORT_TYPE_IB)? + sprintf(type, "%s", (mdev->caps.port_type[info->port] == MLX4_PORT_TYPE_IB)? "ib": "eth"); + if (mdev->caps.possible_type[info->port] == MLX4_PORT_TYPE_AUTO) + sprintf(buf, "auto (%s)\n", type); + else + sprintf(buf, "%s\n", type); + return strlen(buf); } @@ -328,6 +348,8 @@ static ssize_t set_port_type(struct device *dev, info->tmp_type = MLX4_PORT_TYPE_IB; else if (!strcmp(buf, "eth\n")) info->tmp_type = MLX4_PORT_TYPE_ETH; + else if (!strcmp(buf, "auto\n")) + info->tmp_type = MLX4_PORT_TYPE_AUTO; else { mlx4_err(mdev, "%s is not supported port type\n", buf); return -EINVAL; @@ -342,8 +364,13 @@ static ssize_t set_port_type(struct device *dev, if (err) goto out; - for (i = 1; i <= mdev->caps.num_ports; i++) - priv->port[i].tmp_type = 0; + for (i = 0; i < mdev->caps.num_ports; i++) { + mdev->caps.possible_type[i + 1] = types[i]; + if (types[i] == MLX4_PORT_TYPE_AUTO) + types[i] = mdev->caps.port_type[i + 1]; + + priv->port[i + 1].tmp_type = 0; + } err = mlx4_change_port_types(mdev, types); @@ -961,6 +988,31 @@ static void mlx4_cleanup_port_info(struct mlx4_port_info *info) device_remove_file(&info->dev->pdev->dev, &info->port_attr); } +static void mlx4_set_actual_type(struct mlx4_dev *dev) +{ + enum mlx4_port_type stype[dev->caps.num_ports]; + int i; + + if (!(dev->caps.flags & MLX4_DEV_CAP_FLAG_DPDP)) + return; + + for (i = 1; i <= dev->caps.num_ports; i++) { + stype[i-1] = 0; + if (mlx4_priv(dev)->sense.sense_allowed[i] && + dev->caps.possible_type[i] == MLX4_PORT_TYPE_AUTO) { + if (mlx4_SENSE_PORT(dev, i, &stype[i-1])) + return; + } + if (!stype[i-1]) + stype[i-1] = dev->caps.port_type[i]; + } + + if (!mlx4_check_port_params(dev, stype)) { + for (i = 1; i <= dev->caps.num_ports; i++) + dev->caps.port_type[i] = stype[i-1]; + } +} + static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) { struct mlx4_priv *priv; @@ -1084,14 +1136,23 @@ static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id) goto err_port; } - err = mlx4_register_device(dev); + mlx4_set_actual_type(dev); + + err = mlx4_sense_init(dev); if (err) goto err_port; + err = mlx4_register_device(dev); + if (err) + goto err_sense; + pci_set_drvdata(pdev, dev); return 0; +err_sense: + mlx4_sense_cleanup(dev); + err_port: for (port = 1; port <= dev->caps.num_ports; port++) mlx4_cleanup_port_info(&priv->port[port]); @@ -1151,12 +1212,11 @@ static void mlx4_remove_one(struct pci_dev *pdev) if (dev) { mlx4_unregister_device(dev); - + mlx4_sense_cleanup(dev); for (p = 1; p <= dev->caps.num_ports; p++) { mlx4_cleanup_port_info(&priv->port[p]); mlx4_CLOSE_PORT(dev, p); } - mlx4_cleanup_mcg_table(dev); mlx4_cleanup_qp_table(dev); mlx4_cleanup_srq_table(dev); @@ -1213,7 +1273,8 @@ static int __init mlx4_verify_params(void) for (i = 0; i < MLX4_MAX_PORTS; ++i) { if (strcmp(port_type_arr[i], "eth") && - strcmp(port_type_arr[i], "ib")) { + strcmp(port_type_arr[i], "ib") && + strcmp(port_type_arr[i], "auto")) { printk(KERN_WARNING "mlx4_core: bad port_type for " "port %d: %s\n", i, port_type_arr[i]); return -1; diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a235070..8ce983d 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -40,6 +40,8 @@ #include #include #include +#include +#include #include #include @@ -286,6 +288,14 @@ struct mlx4_port_info { struct mlx4_vlan_table vlan_table; }; +struct mlx4_sense { + struct mlx4_dev *dev; + u8 do_sense_port[MLX4_MAX_PORTS + 1]; + u8 sense_allowed[MLX4_MAX_PORTS + 1]; + struct delayed_work sense_poll; + struct workqueue_struct *sense_wq; +}; + struct mlx4_priv { struct mlx4_dev dev; @@ -315,6 +325,7 @@ struct mlx4_priv { struct mlx4_uar driver_uar; void __iomem *kar; struct mlx4_port_info port[MLX4_MAX_PORTS + 1]; + struct mlx4_sense sense; spinlock_t port_lock; }; @@ -323,6 +334,9 @@ static inline struct mlx4_priv *mlx4_priv(struct mlx4_dev *dev) return container_of(dev, struct mlx4_priv, dev); } +#define MIN_SENCE_RANGE HZ +#define MAX_SENCE_RANGE (HZ * 3) + u32 mlx4_bitmap_alloc(struct mlx4_bitmap *bitmap); void mlx4_bitmap_free(struct mlx4_bitmap *bitmap, u32 obj); u32 mlx4_bitmap_alloc_range(struct mlx4_bitmap *bitmap, int cnt, int align); @@ -386,6 +400,16 @@ void mlx4_srq_event(struct mlx4_dev *dev, u32 srqn, int event_type); void mlx4_handle_catas_err(struct mlx4_dev *dev); +void mlx4_start_sense(struct mlx4_dev *dev); +void mlx4_stop_sense(struct mlx4_dev *dev); +int mlx4_sense_init(struct mlx4_dev *dev); +void mlx4_sense_cleanup(struct mlx4_dev *dev); +int mlx4_SENSE_PORT(struct mlx4_dev *dev, int port, enum mlx4_port_type *type); +int mlx4_check_port_params(struct mlx4_dev *dev, + enum mlx4_port_type *port_type); +int mlx4_change_port_types(struct mlx4_dev *dev, + enum mlx4_port_type *port_types); + void mlx4_init_mac_table(struct mlx4_dev *dev, struct mlx4_mac_table *table); void mlx4_init_vlan_table(struct mlx4_dev *dev, struct mlx4_vlan_table *table); diff --git a/drivers/net/mlx4/sense.c b/drivers/net/mlx4/sense.c new file mode 100644 index 0000000..999dcce --- /dev/null +++ b/drivers/net/mlx4/sense.c @@ -0,0 +1,148 @@ +/* + * Copyright (c) 2007 Mellanox Technologies. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#include +#include + +#include + +#include "mlx4.h" + +static inline unsigned long mlx4_gen_sense_tout(void) +{ + return MIN_SENCE_RANGE + random32() % + (MAX_SENCE_RANGE - MIN_SENCE_RANGE); +} + +int mlx4_SENSE_PORT(struct mlx4_dev *dev, int port, enum mlx4_port_type *type) +{ + u64 out_param; + int err = 0; + + err = mlx4_cmd_imm(dev, 0, &out_param, port, 0, + MLX4_CMD_SENSE_PORT, MLX4_CMD_TIME_CLASS_B); + if (err) { + mlx4_err(dev, "Sense command failed for port: %d\n", port); + return err; + } + + if (out_param > 2) { + mlx4_err(dev, "Sense returned illegal value: 0x%llx\n", out_param); + return EINVAL; + } + + *type = out_param; + return 0; +} + +static void mlx4_sense_port(struct work_struct *work) +{ + struct delayed_work *delay = container_of(work, struct delayed_work, work); + struct mlx4_sense *sense = container_of(delay, struct mlx4_sense, + sense_poll); + struct mlx4_dev *dev = sense->dev; + struct mlx4_priv *priv = mlx4_priv(dev); + enum mlx4_port_type stype[dev->caps.num_ports]; + int err = 0; + int i; + + spin_lock(&priv->port_lock); + for (i = 1; i <= dev->caps.num_ports; i++) { + stype[i-1] = 0; + if (sense->do_sense_port[i] && sense->sense_allowed[i] && + dev->caps.possible_type[i] == MLX4_PORT_TYPE_AUTO) { + err = mlx4_SENSE_PORT(dev, i, &stype[i-1]); + if (err) + goto sense_again; + } + if (!stype[i-1]) + stype[i-1] = dev->caps.port_type[i]; + } + + if (mlx4_check_port_params(dev, stype)) + goto sense_again; + + if (mlx4_change_port_types(dev, stype)) + mlx4_err(dev, "Failed to change port_types\n"); + +sense_again: + spin_unlock(&priv->port_lock); + queue_delayed_work(sense->sense_wq , &sense->sense_poll, + mlx4_gen_sense_tout()); +} + + +void mlx4_start_sense(struct mlx4_dev *dev) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + struct mlx4_sense *sense = &priv->sense; + + if (!(dev->caps.flags & MLX4_DEV_CAP_FLAG_DPDP)) + return; + + queue_delayed_work(sense->sense_wq , &sense->sense_poll, + mlx4_gen_sense_tout()); +} + + +void mlx4_stop_sense(struct mlx4_dev *dev) +{ + cancel_delayed_work(&mlx4_priv(dev)->sense.sense_poll); +} + +int mlx4_sense_init(struct mlx4_dev *dev) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + struct mlx4_sense *sense = &priv->sense; + int port; + + sense->dev = dev; + sense->sense_wq = create_singlethread_workqueue("mlx4_sense"); + if (!sense->sense_wq) + return -ENOMEM; + + for (port = 1; port <= dev->caps.num_ports; port++) + sense->do_sense_port[port] = 1; + + INIT_DELAYED_WORK(&sense->sense_poll, mlx4_sense_port); + + return 0; +} + +void mlx4_sense_cleanup(struct mlx4_dev *dev) +{ + mlx4_stop_sense(dev); + flush_workqueue(mlx4_priv(dev)->sense.sense_wq); + destroy_workqueue(mlx4_priv(dev)->sense.sense_wq); +} + diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h index cf9c679..0f82293 100644 --- a/include/linux/mlx4/cmd.h +++ b/include/linux/mlx4/cmd.h @@ -55,6 +55,7 @@ enum { MLX4_CMD_CLOSE_PORT = 0xa, MLX4_CMD_QUERY_HCA = 0xb, MLX4_CMD_QUERY_PORT = 0x43, + MLX4_CMD_SENSE_PORT = 0x4d, MLX4_CMD_SET_PORT = 0xc, MLX4_CMD_ACCESS_DDR = 0x2e, MLX4_CMD_MAP_ICM = 0xffa, diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index d1ecfdc..c63c345 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -145,8 +145,9 @@ enum qp_region { }; enum mlx4_port_type { - MLX4_PORT_TYPE_IB = 1 << 0, - MLX4_PORT_TYPE_ETH = 1 << 1, + MLX4_PORT_TYPE_IB = 1, + MLX4_PORT_TYPE_ETH = 2, + MLX4_PORT_TYPE_AUTO = 3 }; enum { @@ -218,6 +219,7 @@ struct mlx4_caps { int log_num_vlans; int log_num_prios; enum mlx4_port_type port_type[MLX4_MAX_PORTS + 1]; + enum mlx4_port_type possible_type[MLX4_MAX_PORTS + 1]; int reserved_fexch_mpts_base; int num_fexch_mpts; }; -- 1.5.3.7 From yevgenyp at mellanox.co.il Wed Jul 9 06:36:01 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 09 Jul 2008 16:36:01 +0300 Subject: [ofa-general][PATCH 11/11 v3] mlx4_core: Added support to Ethernet device IDs Message-ID: <4874BEC1.5050900@mellanox.co.il> mlx4_core: Added support to Ethernet device IDs Signed-off-by: Yevgeny Petrilin --- drivers/net/mlx4/main.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index e1a3058..9a9c367 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -1256,6 +1256,8 @@ static struct pci_device_id mlx4_pci_table[] = { { PCI_VDEVICE(MELLANOX, 0x6354) }, /* MT25408 "Hermon" QDR */ { PCI_VDEVICE(MELLANOX, 0x6732) }, /* MT25408 "Hermon" DDR PCIe gen2 */ { PCI_VDEVICE(MELLANOX, 0x673c) }, /* MT25408 "Hermon" QDR PCIe gen2 */ + { PCI_VDEVICE(MELLANOX, 0x6368) }, /* MT25408 "Hermon"EN 10GigE */ + { PCI_VDEVICE(MELLANOX, 0x6750) }, /* MT25408 "Hermon"EN 10GigE + Gen2 */ { 0, } }; -- 1.5.3.7 From jean-vincent.ficet at bull.net Wed Jul 9 06:37:57 2008 From: jean-vincent.ficet at bull.net (Vincent Ficet) Date: Wed, 09 Jul 2008 15:37:57 +0200 Subject: [ofa-general] Issues with osm_lin_fwd_tbl.h:osm_lin_fwd_tbl_get_lids_per_block() Message-ID: <4874BF35.4060805@bull.net> Hello, The function osm_lin_fwd_tbl.h:osm_lin_fwd_tbl_get_lids_per_block() is coded as follows: static inline uint16_t osm_lin_fwd_tbl_get_lids_per_block(IN const osm_lin_fwd_tbl_t * const p_tbl) { UNUSED_PARAM(p_tbl); return (64); } Given that the block used comes from the 'data' (payload) field of the ib_smp_t structure (defined in opensm/include/iba/ib_types.h) which is: uint8_t data[IB_SMP_DATA_SIZE]; where IB_SMP_DATA_SIZE is defined as: #define IB_SMP_DATA_SIZE 64 Shouldn't osm_lin_fwd_tbl_get_lids_per_block() be implemented as follows: { UNUSED_PARAM(p_tbl); return IB_SMP_DATA_SIZE; } Also, a LID beeing 16 bit wide, the function name is rather misleading if not wrong: In actual fact, it should rather be called osm_lin_fwd_tbl_get_ports_per_block(), since output ports are encoded as uint8_t in osm_lin_fwd_tbl.h: typedef struct osm_lin_fwdbl { uint16_t size; uint8_t port_tbl[1]; } osm_lin_fwd_tbl_t; I think the confusion here is the same as a previous issue I raised in commit a4934944004586759b3519afa71db5fddebb6541 Do you agree ? If so, I can post a patch if you wish. Finally, what is the point in keeping the unused parameter p_tbl ? Thanks for your help, Vincent From swise at opengridcomputing.com Wed Jul 9 06:48:51 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 08:48:51 -0500 Subject: [ofa-general] [PATCH for-2.6.27 2/2] RDMA/cxgb3: Fixes for zero-stag. In-Reply-To: References: <20080701201502.20859.88618.stgit@dell3.ogc.int> <20080701201710.20859.39200.stgit@dell3.ogc.int> Message-ID: <4874C1C3.7090309@opengridcomputing.com> Roland Dreier wrote: > by the way, this means we have new FW major versions for 2.6.25, 2.6.26 > and 2.6.27. please, please can we get a libcxgb3 that works with FW 5 > through 7 (since I don't think any of the changes affected userspace > anyway, right)?? > Sorry for dragging my feet on this. Yes, changes for 7.0 don't affect user mode. Sooo....the next release of libcxgb3 will support fw versions 5.x-7.x. I'll have this ready hopefully next week latest. Steve. From tom at opengridcomputing.com Wed Jul 9 07:18:24 2008 From: tom at opengridcomputing.com (Tom Tucker) Date: Wed, 09 Jul 2008 09:18:24 -0500 Subject: [ofa-general] Fast Reg Question Message-ID: <4874C8B0.4030004@opengridcomputing.com> Roland: I'm a little perplexed by the fast reg WR definition. The context is that I'm using the Fast Reg verb to map the local memory that is the data source for an RDMA_WRITE. The WR format, however, only takes an rkey. How does this all work when you're using fast reg to map local memory? Does the WR really need the mr pointer, or both the lkey and rkey? The IBTA spec seems to indicate that it needs more information about the MR than just the rkey. Tom From Robert at saq.co.uk Wed Jul 9 07:12:10 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Wed, 9 Jul 2008 15:12:10 +0100 Subject: [ofa-general] Infinband and DRBD Message-ID: Hi Everyone, Apologies if this question is a bit simple but if anyone has any thoughts I'd appreciated them. This is my first Infiniband experiment. I need to clone disk partitions between Linux boxes, theoretically 400-500Mbytes/second writes could be seen at peak (Total for all partitions) but loading will mostly be smallish random writes. I was thinking of just using Centos with the built-in IPOIB support and DRBD but does anyone have any better suggestions? Thanks, Rob The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon at opengridcomputing.com Wed Jul 9 07:34:41 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Wed, 9 Jul 2008 09:34:41 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807082255.01460.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807071440.46502.olaf.kirch@oracle.com> <20080708202755.GB31809@opengridcomputing.com> <200807082255.01460.olaf.kirch@oracle.com> Message-ID: <20080709143440.GA20798@opengridcomputing.com> On Tue, Jul 08, 2008 at 10:55:00PM +0200, Olaf Kirch wrote: > Hi Jon, > > thanks for doing this! I'll merge everything tomorrow, and try > to break it down into a series of nice patches, and it all goes > well I will at least push the bcopy part to my 1.4 tree. > > > This patch reduces the default send_wr and recv_wr to sizes that Chelsio > > can handle, as the sizes before were too large. Optimally, this should > > be determined with a query of the device. I have added that to my todo > > list as well > > Yes, that'll be nice to do properly. I'll think about this a little > tomorrow. After talking it over with Steve, the max_qp_wr returned from ib_query_device should tell the max size of each individual WR queue. I'll write a quick patch to do that and send it out shortly. Thanks, Jon > > > The lguid and fguid cannot be used for iWARP as those structs do not > > exists and give off a nasty illegal pointer deference. I removed the > > references to those, as the appear to only be for debug messages. > > Fine with me. > > > Lastly, this has the port space differentiation (which is the only way > > TCP and iWARP can happily co-exist). > > Okay. > > Olaf > > On Tuesday 08 July 2008 22:27:55 Jon Mason wrote: > > Hey Olaf, > > With the patch below on top of your previous patch, I am able to get it > > working over iWARP (though RDMA is still breaking, but in a new and > > different way). > > > > The patch modifies rds_ib_laddr_check to use rdma_bind_addr to determine > > if it is a IB/iWARP device by creating a temporary rdma_cm_id and > > binding to the address passed in. I'm sure this can be optimized, but > > I'll leave that as a todo for later. > > > > This patch reduces the default send_wr and recv_wr to sizes that Chelsio > > can handle, as the sizes before were too large. Optimally, this should > > be determined with a query of the device. I have added that to my todo > > list as well > > > > The lguid and fguid cannot be used for iWARP as those structs do not > > exists and give off a nasty illegal pointer deference. I removed the > > references to those, as the appear to only be for debug messages. > > > > Lastly, this has the port space differentiation (which is the only way > > TCP and iWARP can happily co-exist). > > > > Let me know what you think. > > > > Thanks, > > Jon > > > > Signed-off-by: Jon Mason > > > > diff --git a/net/rds/ib.c b/net/rds/ib.c > > index cd2dc7c..067fab0 100644 > > --- a/net/rds/ib.c > > +++ b/net/rds/ib.c > > @@ -158,17 +158,29 @@ void rds_ib_remove_one(struct ib_device *device) > > */ > > static int rds_ib_laddr_check(__be32 addr) > > { > > - struct net_device *dev; > > - int ret; > > + int ret = -EADDRNOTAVAIL; > > + struct rdma_cm_id *cm_id; > > + struct sockaddr_in sin; > > + > > + cm_id = rdma_create_id(NULL, NULL, RDMA_PS_TCP); > > + if (cm_id == NULL) { > > + printk("rdma_create_id failed\n"); > > + return -EADDRNOTAVAIL; > > + } > > > > - dev = ip_dev_find(addr); > > - if (dev && dev->type == ARPHRD_INFINIBAND) { > > - dev_put(dev); > > - ret = 0; > > - } else > > + memset(&sin, 0, sizeof(sin)); > > + sin.sin_family = AF_INET; > > + sin.sin_addr.s_addr = addr; > > + > > + /* rdma_bind_addr will fail for non-IB/iWARP devices */ > > + ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin); > > + if (ret) > > ret = -EADDRNOTAVAIL; > > > > - rdsdebug("addr %u.%u.%u.%u ret %d\n", NIPQUAD(addr), ret); > > + rdsdebug("addr %u.%u.%u.%u ret %d node type %d\n", > > + NIPQUAD(addr), ret, cm_id->device?cm_id->device->node_type:-1); > > + > > + rdma_destroy_id(cm_id); > > > > return ret; > > } > > diff --git a/net/rds/ib.h b/net/rds/ib.h > > index 947977c..2924fda 100644 > > --- a/net/rds/ib.h > > +++ b/net/rds/ib.h > > @@ -7,14 +7,16 @@ > > > > #define RDS_IB_RESOLVE_TIMEOUT_MS 5000 > > > > -#define RDS_FMR_SIZE 256 > > +/* FIXME - use ib_query_device to determine proper value */ > > +#define RDS_FMR_SIZE 20 > > #define RDS_FMR_POOL_SIZE 2048 > > > > -#define RDS_IB_MAX_SGE 8 > > +#define RDS_IB_MAX_SGE 8 > > #define RDS_IB_RECV_SGE 2 > > > > -#define RDS_IB_DEFAULT_RECV_WR 1024 > > -#define RDS_IB_DEFAULT_SEND_WR 256 > > +/* FIXME - call ib_query_device to determine sane values for this based on the HW */ > > +#define RDS_IB_DEFAULT_RECV_WR 512 > > +#define RDS_IB_DEFAULT_SEND_WR 128 > > > > #define RDS_IB_SUPPORTED_PROTOCOLS 0x00000003 /* minor versions supported */ > > > > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > > index a49e394..fd56481 100644 > > --- a/net/rds/ib_cm.c > > +++ b/net/rds/ib_cm.c > > @@ -296,7 +296,7 @@ static int rds_ib_setup_qp(struct rds_connection *conn) > > */ > > ret = rdma_create_qp(ic->i_cm_id, ic->i_pd, &attr); > > if (ret) { > > - rdsdebug("ib_req_notify_cq failed: %d\n", ret); > > + rdsdebug("rdma_create_qp failed: %d\n", ret); > > goto out; > > } > > > > @@ -388,8 +388,6 @@ static u32 rds_ib_protocol_compatible(const struct rds_ib_connect_private *dp) > > static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, > > struct rdma_cm_event *event) > > { > > - __be64 lguid = cm_id->route.path_rec->sgid.global.interface_id; > > - __be64 fguid = cm_id->route.path_rec->dgid.global.interface_id; > > const struct rds_ib_connect_private *dp = event->param.conn.private_data; > > struct rds_ib_connect_private dp_rep; > > struct rds_connection *conn = NULL; > > @@ -404,11 +402,9 @@ static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, > > if (!version) > > goto out; > > > > - rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u lguid 0x%llx fguid " > > - "0x%llx\n", NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), > > - RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version), > > - (unsigned long long)be64_to_cpu(lguid), > > - (unsigned long long)be64_to_cpu(fguid)); > > + rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u\n", > > + NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), > > + RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version)); > > > > conn = rds_conn_create(dp->dp_daddr, dp->dp_saddr, &rds_ib_transport, > > GFP_KERNEL); > > @@ -628,7 +624,7 @@ int rds_ib_conn_connect(struct rds_connection *conn) > > > > dest.sin_family = AF_INET; > > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > > - dest.sin_port = (__force u16)htons(RDS_PORT); > > + dest.sin_port = (__force u16)htons(RDS_IB_PORT); > > > > ret = rdma_resolve_addr(ic->i_cm_id, (struct sockaddr *)&src, > > (struct sockaddr *)&dest, > > @@ -813,7 +809,7 @@ int __init rds_ib_listen_init(void) > > > > sin.sin_family = PF_INET, > > sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); > > - sin.sin_port = (__force u16)htons(RDS_PORT); > > + sin.sin_port = (__force u16)htons(RDS_IB_PORT); > > > > /* > > * XXX I bet this binds the cm_id to a device. If we want to support > > @@ -833,7 +829,7 @@ int __init rds_ib_listen_init(void) > > goto out; > > } > > > > - rdsdebug("cm %p listening on port %u\n", cm_id, RDS_PORT); > > + rdsdebug("cm %p listening on port %u\n", cm_id, RDS_IB_PORT); > > > > rds_ib_listen_id = cm_id; > > cm_id = NULL; > > diff --git a/net/rds/rds.h b/net/rds/rds.h > > index 03031e2..6c17a4d 100644 > > --- a/net/rds/rds.h > > +++ b/net/rds/rds.h > > @@ -26,8 +26,8 @@ > > * > > * port 18633 was the version that had ack frames on the wire. > > */ > > -#define RDS_PORT 18634 > > - > > +#define RDS_TCP_PORT 18634 > > +#define RDS_IB_PORT 18635 > > > > #ifndef AF_RDS > > #define AF_RDS 28 /* Reliable Datagram Socket */ > > diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c > > index 0389a99..298e372 100644 > > --- a/net/rds/tcp_connect.c > > +++ b/net/rds/tcp_connect.c > > @@ -96,7 +96,7 @@ int rds_tcp_conn_connect(struct rds_connection *conn) > > > > dest.sin_family = AF_INET; > > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > > - dest.sin_port = (__force u16)htons(RDS_PORT); > > + dest.sin_port = (__force u16)htons(RDS_TCP_PORT); > > > > /* > > * once we call connect() we can start getting callbacks and they > > diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c > > index caeacbe..50709b7 100644 > > --- a/net/rds/tcp_listen.c > > +++ b/net/rds/tcp_listen.c > > @@ -159,7 +159,7 @@ int __init rds_tcp_listen_init(void) > > > > sin.sin_family = PF_INET, > > sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); > > - sin.sin_port = (__force u16)htons(RDS_PORT); > > + sin.sin_port = (__force u16)htons(RDS_TCP_PORT); > > > > ret = sock->ops->bind(sock, (struct sockaddr *)&sin, sizeof(sin)); > > if (ret < 0) > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > > > -- > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From sashak at voltaire.com Wed Jul 9 07:35:40 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 17:35:40 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <48749387.2060706@dev.mellanox.co.il> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> <4874546E.8000108@mellanox.co.il> <20080709061701.GI12551@sashak.voltaire.com> <487487FE.1080406@mellanox.co.il> <48749387.2060706@dev.mellanox.co.il> Message-ID: <20080709143540.GK12551@sashak.voltaire.com> Hi Oren, On 13:31 Wed 09 Jul , Oren Kladnitsky wrote: > No. Ibutils use opensm libvendor for mad sending interface. Right, it is not used directly. But it is used in LDFLAGS (due to libosmvendor -> libibumad -> libibcommon dependency): config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor -losmcomp -libumad -libcommon" ibis/config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor -losmcomp -libumad -libcommon" ibmgtsim/config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor -losmcomp -libumad -libcommon" Sasha From kliteyn at dev.mellanox.co.il Wed Jul 9 07:54:47 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 09 Jul 2008 17:54:47 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <20080709143540.GK12551@sashak.voltaire.com> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> <4874546E.8000108@mellanox.co.il> <20080709061701.GI12551@sashak.voltaire.com> <487487FE.1080406@mellanox.co.il> <48749387.2060706@dev.mellanox.co.il> <20080709143540.GK12551@sashak.voltaire.com> Message-ID: <4874D137.8050303@dev.mellanox.co.il> Sasha Khapyorsky wrote: > Hi Oren, > > On 13:31 Wed 09 Jul , Oren Kladnitsky wrote: >> No. Ibutils use opensm libvendor for mad sending interface. > > Right, it is not used directly. But it is used in LDFLAGS (due to > libosmvendor -> libibumad -> libibcommon dependency): Right, but there's no reason to leave it as "public". libibumad is public, libibcommon isn't -- Yevgeny > config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor -losmcomp -libumad -libcommon" > ibis/config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor -losmcomp -libumad -libcommon" > ibmgtsim/config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor -losmcomp -libumad -libcommon" > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From olaf.kirch at oracle.com Wed Jul 9 07:57:08 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Wed, 9 Jul 2008 16:57:08 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080709143440.GA20798@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807082255.01460.olaf.kirch@oracle.com> <20080709143440.GA20798@opengridcomputing.com> Message-ID: <200807091657.09593.olaf.kirch@oracle.com> On Wednesday 09 July 2008 16:34:41 Jon Mason wrote: > After talking it over with Steve, the max_qp_wr returned from > ib_query_device should tell the max size of each individual WR queue. > I'll write a quick patch to do that and send it out shortly. Don't worry, I have that already. I'll push my patches to my git tree in an hour or so. Olaf > > Thanks, > Jon > > > > > > The lguid and fguid cannot be used for iWARP as those structs do not > > > exists and give off a nasty illegal pointer deference. I removed the > > > references to those, as the appear to only be for debug messages. > > > > Fine with me. > > > > > Lastly, this has the port space differentiation (which is the only way > > > TCP and iWARP can happily co-exist). > > > > Okay. > > > > Olaf > > > > On Tuesday 08 July 2008 22:27:55 Jon Mason wrote: > > > Hey Olaf, > > > With the patch below on top of your previous patch, I am able to get it > > > working over iWARP (though RDMA is still breaking, but in a new and > > > different way). > > > > > > The patch modifies rds_ib_laddr_check to use rdma_bind_addr to determine > > > if it is a IB/iWARP device by creating a temporary rdma_cm_id and > > > binding to the address passed in. I'm sure this can be optimized, but > > > I'll leave that as a todo for later. > > > > > > This patch reduces the default send_wr and recv_wr to sizes that Chelsio > > > can handle, as the sizes before were too large. Optimally, this should > > > be determined with a query of the device. I have added that to my todo > > > list as well > > > > > > The lguid and fguid cannot be used for iWARP as those structs do not > > > exists and give off a nasty illegal pointer deference. I removed the > > > references to those, as the appear to only be for debug messages. > > > > > > Lastly, this has the port space differentiation (which is the only way > > > TCP and iWARP can happily co-exist). > > > > > > Let me know what you think. > > > > > > Thanks, > > > Jon > > > > > > Signed-off-by: Jon Mason > > > > > > diff --git a/net/rds/ib.c b/net/rds/ib.c > > > index cd2dc7c..067fab0 100644 > > > --- a/net/rds/ib.c > > > +++ b/net/rds/ib.c > > > @@ -158,17 +158,29 @@ void rds_ib_remove_one(struct ib_device *device) > > > */ > > > static int rds_ib_laddr_check(__be32 addr) > > > { > > > - struct net_device *dev; > > > - int ret; > > > + int ret = -EADDRNOTAVAIL; > > > + struct rdma_cm_id *cm_id; > > > + struct sockaddr_in sin; > > > + > > > + cm_id = rdma_create_id(NULL, NULL, RDMA_PS_TCP); > > > + if (cm_id == NULL) { > > > + printk("rdma_create_id failed\n"); > > > + return -EADDRNOTAVAIL; > > > + } > > > > > > - dev = ip_dev_find(addr); > > > - if (dev && dev->type == ARPHRD_INFINIBAND) { > > > - dev_put(dev); > > > - ret = 0; > > > - } else > > > + memset(&sin, 0, sizeof(sin)); > > > + sin.sin_family = AF_INET; > > > + sin.sin_addr.s_addr = addr; > > > + > > > + /* rdma_bind_addr will fail for non-IB/iWARP devices */ > > > + ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin); > > > + if (ret) > > > ret = -EADDRNOTAVAIL; > > > > > > - rdsdebug("addr %u.%u.%u.%u ret %d\n", NIPQUAD(addr), ret); > > > + rdsdebug("addr %u.%u.%u.%u ret %d node type %d\n", > > > + NIPQUAD(addr), ret, cm_id->device?cm_id->device->node_type:-1); > > > + > > > + rdma_destroy_id(cm_id); > > > > > > return ret; > > > } > > > diff --git a/net/rds/ib.h b/net/rds/ib.h > > > index 947977c..2924fda 100644 > > > --- a/net/rds/ib.h > > > +++ b/net/rds/ib.h > > > @@ -7,14 +7,16 @@ > > > > > > #define RDS_IB_RESOLVE_TIMEOUT_MS 5000 > > > > > > -#define RDS_FMR_SIZE 256 > > > +/* FIXME - use ib_query_device to determine proper value */ > > > +#define RDS_FMR_SIZE 20 > > > #define RDS_FMR_POOL_SIZE 2048 > > > > > > -#define RDS_IB_MAX_SGE 8 > > > +#define RDS_IB_MAX_SGE 8 > > > #define RDS_IB_RECV_SGE 2 > > > > > > -#define RDS_IB_DEFAULT_RECV_WR 1024 > > > -#define RDS_IB_DEFAULT_SEND_WR 256 > > > +/* FIXME - call ib_query_device to determine sane values for this based on the HW */ > > > +#define RDS_IB_DEFAULT_RECV_WR 512 > > > +#define RDS_IB_DEFAULT_SEND_WR 128 > > > > > > #define RDS_IB_SUPPORTED_PROTOCOLS 0x00000003 /* minor versions supported */ > > > > > > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > > > index a49e394..fd56481 100644 > > > --- a/net/rds/ib_cm.c > > > +++ b/net/rds/ib_cm.c > > > @@ -296,7 +296,7 @@ static int rds_ib_setup_qp(struct rds_connection *conn) > > > */ > > > ret = rdma_create_qp(ic->i_cm_id, ic->i_pd, &attr); > > > if (ret) { > > > - rdsdebug("ib_req_notify_cq failed: %d\n", ret); > > > + rdsdebug("rdma_create_qp failed: %d\n", ret); > > > goto out; > > > } > > > > > > @@ -388,8 +388,6 @@ static u32 rds_ib_protocol_compatible(const struct rds_ib_connect_private *dp) > > > static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, > > > struct rdma_cm_event *event) > > > { > > > - __be64 lguid = cm_id->route.path_rec->sgid.global.interface_id; > > > - __be64 fguid = cm_id->route.path_rec->dgid.global.interface_id; > > > const struct rds_ib_connect_private *dp = event->param.conn.private_data; > > > struct rds_ib_connect_private dp_rep; > > > struct rds_connection *conn = NULL; > > > @@ -404,11 +402,9 @@ static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, > > > if (!version) > > > goto out; > > > > > > - rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u lguid 0x%llx fguid " > > > - "0x%llx\n", NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), > > > - RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version), > > > - (unsigned long long)be64_to_cpu(lguid), > > > - (unsigned long long)be64_to_cpu(fguid)); > > > + rdsdebug("saddr %u.%u.%u.%u daddr %u.%u.%u.%u RDSv%u.%u\n", > > > + NIPQUAD(dp->dp_saddr), NIPQUAD(dp->dp_daddr), > > > + RDS_PROTOCOL_MAJOR(version), RDS_PROTOCOL_MINOR(version)); > > > > > > conn = rds_conn_create(dp->dp_daddr, dp->dp_saddr, &rds_ib_transport, > > > GFP_KERNEL); > > > @@ -628,7 +624,7 @@ int rds_ib_conn_connect(struct rds_connection *conn) > > > > > > dest.sin_family = AF_INET; > > > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > > > - dest.sin_port = (__force u16)htons(RDS_PORT); > > > + dest.sin_port = (__force u16)htons(RDS_IB_PORT); > > > > > > ret = rdma_resolve_addr(ic->i_cm_id, (struct sockaddr *)&src, > > > (struct sockaddr *)&dest, > > > @@ -813,7 +809,7 @@ int __init rds_ib_listen_init(void) > > > > > > sin.sin_family = PF_INET, > > > sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); > > > - sin.sin_port = (__force u16)htons(RDS_PORT); > > > + sin.sin_port = (__force u16)htons(RDS_IB_PORT); > > > > > > /* > > > * XXX I bet this binds the cm_id to a device. If we want to support > > > @@ -833,7 +829,7 @@ int __init rds_ib_listen_init(void) > > > goto out; > > > } > > > > > > - rdsdebug("cm %p listening on port %u\n", cm_id, RDS_PORT); > > > + rdsdebug("cm %p listening on port %u\n", cm_id, RDS_IB_PORT); > > > > > > rds_ib_listen_id = cm_id; > > > cm_id = NULL; > > > diff --git a/net/rds/rds.h b/net/rds/rds.h > > > index 03031e2..6c17a4d 100644 > > > --- a/net/rds/rds.h > > > +++ b/net/rds/rds.h > > > @@ -26,8 +26,8 @@ > > > * > > > * port 18633 was the version that had ack frames on the wire. > > > */ > > > -#define RDS_PORT 18634 > > > - > > > +#define RDS_TCP_PORT 18634 > > > +#define RDS_IB_PORT 18635 > > > > > > #ifndef AF_RDS > > > #define AF_RDS 28 /* Reliable Datagram Socket */ > > > diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c > > > index 0389a99..298e372 100644 > > > --- a/net/rds/tcp_connect.c > > > +++ b/net/rds/tcp_connect.c > > > @@ -96,7 +96,7 @@ int rds_tcp_conn_connect(struct rds_connection *conn) > > > > > > dest.sin_family = AF_INET; > > > dest.sin_addr.s_addr = (__force u32)conn->c_faddr; > > > - dest.sin_port = (__force u16)htons(RDS_PORT); > > > + dest.sin_port = (__force u16)htons(RDS_TCP_PORT); > > > > > > /* > > > * once we call connect() we can start getting callbacks and they > > > diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c > > > index caeacbe..50709b7 100644 > > > --- a/net/rds/tcp_listen.c > > > +++ b/net/rds/tcp_listen.c > > > @@ -159,7 +159,7 @@ int __init rds_tcp_listen_init(void) > > > > > > sin.sin_family = PF_INET, > > > sin.sin_addr.s_addr = (__force u32)htonl(INADDR_ANY); > > > - sin.sin_port = (__force u16)htons(RDS_PORT); > > > + sin.sin_port = (__force u16)htons(RDS_TCP_PORT); > > > > > > ret = sock->ops->bind(sock, (struct sockaddr *)&sin, sizeof(sin)); > > > if (ret < 0) > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > > > > > > > > -- > > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From chu11 at llnl.gov Wed Jul 9 08:50:11 2008 From: chu11 at llnl.gov (Al Chu) Date: Wed, 09 Jul 2008 08:50:11 -0700 Subject: [ofa-general] Re: [OPENSM] [PATCH] reroute console option In-Reply-To: <20080709014503.GL26875@sashak.voltaire.com> References: <1215465953.16508.58.camel@cardanus.llnl.gov> <20080709014503.GL26875@sashak.voltaire.com> Message-ID: <1215618611.16508.98.camel@cardanus.llnl.gov> Hey Sasha, On Wed, 2008-07-09 at 04:45 +0300, Sasha Khapyorsky wrote: > Hi Al, > > On 14:25 Mon 07 Jul , Al Chu wrote: > > > > With all of these file-input possibilities for OpenSM routing, I don't > > think there is a way to tell OpenSM to reroute if (for example) the file > > input has changed but the network/fabric has not. > > Yes, but you can request a heavy sweep with SIGHUP or from console. Yup. I forgot to append a "without requiring a heavy sweep" to the above statement :-) > > > This is a patch to > > add a command to the console to tell the console to reroute but not do a > > heavy sweep. > > [snip...] > > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > > index 87d6d5b..0501aa2 100644 > > --- a/opensm/opensm/osm_state_mgr.c > > +++ b/opensm/opensm/osm_state_mgr.c > > @@ -1020,6 +1020,7 @@ static void do_sweep(osm_sm_t * sm) > > && sm->p_subn->sm_state != IB_SMINFO_STATE_DISCOVERING > > && sm->p_subn->opt.force_heavy_sweep == FALSE > > && sm->p_subn->force_heavy_sweep == FALSE > > + && sm->p_subn->force_reroute == FALSE > > && sm->p_subn->subnet_initialization_error == FALSE > > && (__osm_state_mgr_light_sweep_start(sm) == IB_SUCCESS)) { > > if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) > > @@ -1031,11 +1032,43 @@ static void do_sweep(osm_sm_t * sm) > > } > > } > > > > + /* > > + * If we don't need to do a heavy sweep and we want to do a reroute, > > + * just reroute only. > > + */ > > + if (cl_qmap_count(&sm->p_subn->sw_guid_tbl) > > + && sm->p_subn->sm_state != IB_SMINFO_STATE_DISCOVERING > > + && sm->p_subn->opt.force_heavy_sweep == FALSE > > + && sm->p_subn->force_heavy_sweep == FALSE > > + && sm->p_subn->force_reroute == TRUE > > + && sm->p_subn->subnet_initialization_error == FALSE) { > > + /* Reset flag */ > > + sm->p_subn->force_reroute = FALSE; > > + > > + /* Re-program the switches fully */ > > + sm->p_subn->ignore_existing_lfts = TRUE; > > + > > + osm_ucast_mgr_process(&sm->ucast_mgr); > > + > > + /* Reset flag */ > > + sm->p_subn->ignore_existing_lfts = FALSE; > > + > > + if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) > > + return; > > + > > + if (!sm->p_subn->force_heavy_sweep) { > > I cannot see how 'force_heavy_sweep' can be triggered by > osm_ucast_mgr_process(), but I do see that 'subnet_initialization_error' > can be set to TRUE during re-routing. So it looks that we need to check > this flag instead. I added the force-heavy-sweep check more as a safeguard for any future changes. I'll add in the 'subnet_initialization_error' check instead. Al > Sasha > > > + osm_log_msg_box(sm->p_log, OSM_LOG_VERBOSE, > > + __FUNCTION__, "REROUTE COMPLETE"); > > + return; > > + } > > + } > > + > > /* go to heavy sweep */ > > _repeat_discovery: > > > > /* First of all - unset all flags */ > > sm->p_subn->force_heavy_sweep = FALSE; > > + sm->p_subn->force_reroute = FALSE; > > sm->p_subn->subnet_initialization_error = FALSE; > > > > /* rescan configuration updates */ > > -- > > 1.5.4.5 > > > -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From olaf.kirch at oracle.com Wed Jul 9 09:15:25 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Wed, 9 Jul 2008 18:15:25 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080709143440.GA20798@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807082255.01460.olaf.kirch@oracle.com> <20080709143440.GA20798@opengridcomputing.com> Message-ID: <200807091815.25992.olaf.kirch@oracle.com> Jon, please check out git://git.openfabrics.org/~okir/ofed_1_4/linux-2.6.git code-drop-20080703 It has the bcopy changes, and some of the plumbing for RDMA. The remaining bits aren't complete yet. I'm attaching a patch that contains my current working state - this doesn't work, and probably doesn't even compile, but it's for your review. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax -------------- next part -------------- A non-text attachment was scrubbed... Name: rds-ib-fastreg-mrs.patch Type: text/x-patch Size: 28422 bytes Desc: not available URL: From chu11 at llnl.gov Wed Jul 9 09:26:56 2008 From: chu11 at llnl.gov (Al Chu) Date: Wed, 09 Jul 2008 09:26:56 -0700 Subject: [ofa-general] Re: [OPENSM] [PATCH] reroute console option In-Reply-To: <1215618611.16508.98.camel@cardanus.llnl.gov> References: <1215465953.16508.58.camel@cardanus.llnl.gov> <20080709014503.GL26875@sashak.voltaire.com> <1215618611.16508.98.camel@cardanus.llnl.gov> Message-ID: <1215620816.16508.101.camel@cardanus.llnl.gov> Hey Sasha, Attached is the modified patch that checks for subnet_initialization_error. Al On Wed, 2008-07-09 at 08:50 -0700, Al Chu wrote: > Hey Sasha, > > On Wed, 2008-07-09 at 04:45 +0300, Sasha Khapyorsky wrote: > > Hi Al, > > > > On 14:25 Mon 07 Jul , Al Chu wrote: > > > > > > With all of these file-input possibilities for OpenSM routing, I don't > > > think there is a way to tell OpenSM to reroute if (for example) the file > > > input has changed but the network/fabric has not. > > > > Yes, but you can request a heavy sweep with SIGHUP or from console. > > Yup. I forgot to append a "without requiring a heavy sweep" to the > above statement :-) > > > > > > This is a patch to > > > add a command to the console to tell the console to reroute but not do a > > > heavy sweep. > > > > [snip...] > > > > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > > > index 87d6d5b..0501aa2 100644 > > > --- a/opensm/opensm/osm_state_mgr.c > > > +++ b/opensm/opensm/osm_state_mgr.c > > > @@ -1020,6 +1020,7 @@ static void do_sweep(osm_sm_t * sm) > > > && sm->p_subn->sm_state != IB_SMINFO_STATE_DISCOVERING > > > && sm->p_subn->opt.force_heavy_sweep == FALSE > > > && sm->p_subn->force_heavy_sweep == FALSE > > > + && sm->p_subn->force_reroute == FALSE > > > && sm->p_subn->subnet_initialization_error == FALSE > > > && (__osm_state_mgr_light_sweep_start(sm) == IB_SUCCESS)) { > > > if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) > > > @@ -1031,11 +1032,43 @@ static void do_sweep(osm_sm_t * sm) > > > } > > > } > > > > > > + /* > > > + * If we don't need to do a heavy sweep and we want to do a reroute, > > > + * just reroute only. > > > + */ > > > + if (cl_qmap_count(&sm->p_subn->sw_guid_tbl) > > > + && sm->p_subn->sm_state != IB_SMINFO_STATE_DISCOVERING > > > + && sm->p_subn->opt.force_heavy_sweep == FALSE > > > + && sm->p_subn->force_heavy_sweep == FALSE > > > + && sm->p_subn->force_reroute == TRUE > > > + && sm->p_subn->subnet_initialization_error == FALSE) { > > > + /* Reset flag */ > > > + sm->p_subn->force_reroute = FALSE; > > > + > > > + /* Re-program the switches fully */ > > > + sm->p_subn->ignore_existing_lfts = TRUE; > > > + > > > + osm_ucast_mgr_process(&sm->ucast_mgr); > > > + > > > + /* Reset flag */ > > > + sm->p_subn->ignore_existing_lfts = FALSE; > > > + > > > + if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) > > > + return; > > > + > > > + if (!sm->p_subn->force_heavy_sweep) { > > > > I cannot see how 'force_heavy_sweep' can be triggered by > > osm_ucast_mgr_process(), but I do see that 'subnet_initialization_error' > > can be set to TRUE during re-routing. So it looks that we need to check > > this flag instead. > > I added the force-heavy-sweep check more as a safeguard for any future > changes. I'll add in the 'subnet_initialization_error' check instead. > > Al > > > Sasha > > > > > + osm_log_msg_box(sm->p_log, OSM_LOG_VERBOSE, > > > + __FUNCTION__, "REROUTE COMPLETE"); > > > + return; > > > + } > > > + } > > > + > > > /* go to heavy sweep */ > > > _repeat_discovery: > > > > > > /* First of all - unset all flags */ > > > sm->p_subn->force_heavy_sweep = FALSE; > > > + sm->p_subn->force_reroute = FALSE; > > > sm->p_subn->subnet_initialization_error = FALSE; > > > > > > /* rescan configuration updates */ > > > -- > > > 1.5.4.5 > > > > > -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-reroute-console-command.patch Type: text/x-patch Size: 4509 bytes Desc: not available URL: From orenk at dev.mellanox.co.il Wed Jul 9 09:44:52 2008 From: orenk at dev.mellanox.co.il (Oren Kladnitsky) Date: Wed, 09 Jul 2008 19:44:52 +0300 Subject: [ofa-general] Re: [ewg] List of libraries in OFED In-Reply-To: <4874D137.8050303@dev.mellanox.co.il> References: <40FA0A8088E8A441973D37502F00933E3F80FC@mtlexch01.mtl.com> <20080709035616.GB12551@sashak.voltaire.com> <4874546E.8000108@mellanox.co.il> <20080709061701.GI12551@sashak.voltaire.com> <487487FE.1080406@mellanox.co.il> <48749387.2060706@dev.mellanox.co.il> <20080709143540.GK12551@sashak.voltaire.com> <4874D137.8050303@dev.mellanox.co.il> Message-ID: <4874EB04.2050106@dev.mellanox.co.il> Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: >> Hi Oren, >> >> On 13:31 Wed 09 Jul , Oren Kladnitsky wrote: >>> No. Ibutils use opensm libvendor for mad sending interface. >> >> Right, it is not used directly. But it is used in LDFLAGS (due to >> libosmvendor -> libibumad -> libibcommon dependency): > > Right, but there's no reason to leave it as "public". > libibumad is public, libibcommon isn't > > -- Yevgeny > >> config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm -losmvendor >> -losmcomp -libumad -libcommon" >> ibis/config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm >> -losmvendor -losmcomp -libumad -libcommon" >> ibmgtsim/config/osm.m4: OSM_LDFLAGS="$OSM_LDFLAGS -lopensm >> -losmvendor -losmcomp -libumad -libcommon" >> >> Sasha >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > Of course - Let me rephrase: No problem for ibutils with removing libibcommon - just tell me when you want to remove it and I'll remove it from the LDFLAGS libs. From info.lgc at comcast.net Wed Jul 9 09:41:30 2008 From: info.lgc at comcast.net (Lg House) Date: Wed, 09 Jul 2008 16:41:30 +0000 Subject: [ofa-general] =?iso-8859-1?q?Winner_=A9Lg-Life_is_Good?= Message-ID: <070920081641.19413.4874EA3A0000B2CA00004BD522007343640C0904D201000207@comcast.net> Winner �Lg-Life is Good This is to inform you about the unexpected luck your e-mail have brought to you.You have been approved to claim a total sum of 700,000GBP awarded by the Lg Electronics Company. Your email was selected randomly from over 21,000 email Id attach to a ticket numbers 1/8/09/11 and serial number 8-1-7-3GF which consiquently draw the ref number LGE/uk/09/01 that amerge a lucky winner of the 700,000GBP. This program organized by the Lg Electronics Company, is to promote the use of LG products such as Mobile phone,Air Condition,Refrigerator,laptop,microwave, Television,etc as a global village around the world.send your ticket,serial and ref number when contacting the above claims officer. For the remittance of your prize you are to contact our UK Administrative Officer below: Life is Good Prize Administrator Mr James Keegan Esq E-mail: JAMESKEEGANESQ at GMAIL.COM PHONE#: +44703196-9542 Sincerely, Dr.Dawnson mcwright Ramah Winner �Lg-Life is Good This is to inform you about the unexpected luck your e-mail have brought to you.You have been approved to claim a total sum of 700,000GBP awarded by the Lg Electronics Company. Your email was selected randomly from over 21,000 email Id attach to a ticket numbers 1/8/09/11 and serial number 8-1-7-3GF which consiquently draw the ref number LGE/uk/09/01 that amerge a lucky winner of the 700,000GBP. This program organized by the Lg Electronics Company, is to promote the use of LG products such as Mobile phone,Air Condition,Refrigerator,laptop,microwave, Television,etc as a global village around the world.send your ticket,serial and ref number when contacting the above claims officer. For the remittance of your prize you are to contact our UK Administrative Officer below: Life is Good Prize Administrator Mr James Keegan Esq E-mail: JAMESKEEGANESQ at GMAIL.COM PHONE#: +44703196-9542 Sincerely, Dr.Dawnson Mcwright Ramah Winner �Lg-Life is Good This is to inform you about the unexpected luck your e-mail have brought to you.You have been approved to claim a total sum of 700,000GBP awarded by the Lg Electronics Company. Your email was selected randomly from over 21,000 email Id attach to a ticket numbers 1/8/09/11 and serial number 8-1-7-3GF which consiquently draw the ref number LGE/uk/09/01 that amerge a lucky winner of the 700,000GBP. This program organized by the Lg Electronics Company, is to promote the use of LG products such as Mobile phone,Air Condition,Refrigerator,laptop,microwave, Television,etc as a global village around the world.send your ticket,serial and ref number when contacting the above claims officer. For the remittance of your prize you are to contact our UK Administrative Officer below: Life is Good Prize Administrator Mr James Keegan Esq E-mail: JAMESKEEGANESQ at GMAIL.COM PHONE#: +44703196-9542 Sincerely, Dr.Dawnson mcwright Ramah Winner �Lg-Life is Good This is to inform you about the unexpected luck your e-mail have brought to you.You have been approved to claim a total sum of 700,000GBP awarded by the Lg Electronics Company. Your email was selected randomly from over 21,000 email Id attach to a ticket numbers 1/8/09/11 and serial number 8-1-7-3GF which consiquently draw the ref number LGE/uk/09/01 that amerge a lucky winner of the 700,000GBP. This program organized by the Lg Electronics Company, is to promote the use of LG products such as Mobile phone,Air Condition,Refrigerator,laptop,microwave, Television,etc as a global village around the world.send your ticket,serial and ref number when contacting the above claims officer. For the remittance of your prize you are to contact our UK Administrative Officer below: Life is Good Prize Administrator Mr James Keegan Esq E-mail: JAMESKEEGANESQ at GMAIL.COM PHONE#: +44703196-9542 Sincerely, Dr.Dawnson Mcwright Ramah From bryan.d.green at nasa.gov Wed Jul 9 09:55:22 2008 From: bryan.d.green at nasa.gov (Bryan Green) Date: Wed, 09 Jul 2008 09:55:22 -0700 Subject: [ofa-general] tarfiles for libehca, libsdp? Message-ID: <20080709165522.9760B1C8E49@ece06.nas.nasa.gov> Hello, >From where can I download the tar files for libehca and libsdp? They are not available under the http://www.openfabrics.org/downloads/ directory. Thanks, -bryan From swise at opengridcomputing.com Wed Jul 9 10:07:23 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 12:07:23 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807091815.25992.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807082255.01460.olaf.kirch@oracle.com> <20080709143440.GA20798@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> Message-ID: <4874F04B.20603@opengridcomputing.com> Olaf Kirch wrote: > Jon, > > please check out > git://git.openfabrics.org/~okir/ofed_1_4/linux-2.6.git code-drop-20080703 > > It has the bcopy changes, and some of the plumbing for RDMA. The > remaining bits aren't complete yet. I'm attaching a patch that contains > my current working state - this doesn't work, and probably doesn't even > compile, but it's for your review. > > Olaf > They definitely won't compile in the ofed_1_4 tree as it stands. The MEM_MGT_EXTENSIONS patches are still in roland's for-2.6.27 branch. Jon's development work to date has been on ofed-1.3.1 + patches I made to include this functionality. I've been hesitant to create similar patches for ofed-1.4 since it will be merged up to 2.6.27-rc1 which will have all this stuff. From sashak at voltaire.com Wed Jul 9 10:13:58 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 20:13:58 +0300 Subject: [ofa-general] Re: [OPENSM] [PATCH] reroute console option In-Reply-To: <1215620816.16508.101.camel@cardanus.llnl.gov> References: <1215465953.16508.58.camel@cardanus.llnl.gov> <20080709014503.GL26875@sashak.voltaire.com> <1215618611.16508.98.camel@cardanus.llnl.gov> <1215620816.16508.101.camel@cardanus.llnl.gov> Message-ID: <20080709171357.GL12551@sashak.voltaire.com> On 09:26 Wed 09 Jul , Al Chu wrote: > From 5ce5da8463b1e7ddfbc03c878bfbef28022ae652 Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Fri, 27 Jun 2008 10:38:47 -0700 > Subject: [PATCH] add reroute console command > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From sashak at voltaire.com Wed Jul 9 10:26:22 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 20:26:22 +0300 Subject: [ofa-general] I will be on vacation Jule 10-22 Message-ID: <20080709172622.GM12551@sashak.voltaire.com> Hi, I will be on vacation Jule 10-22 and likely "off-line" most of the period. IB management daily build is working in automatic mode :) Sasha From olaf.kirch at oracle.com Wed Jul 9 10:38:15 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Wed, 9 Jul 2008 19:38:15 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <4874F04B.20603@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> <4874F04B.20603@opengridcomputing.com> Message-ID: <200807091938.15779.olaf.kirch@oracle.com> On Wednesday 09 July 2008 19:07:23 Steve Wise wrote: > They definitely won't compile in the ofed_1_4 tree as it stands. The > MEM_MGT_EXTENSIONS patches are still in roland's for-2.6.27 branch. That's why I included these patches in my test branch :-) For the record: everything on this git branch compiles for me. It's just the extra RDMA patch, which isn't in the git repo, that doesn't. > Jon's development work to date has been on ofed-1.3.1 + patches I made > to include this functionality. I've been hesitant to create similar > patches for ofed-1.4 since it will be merged up to 2.6.27-rc1 which will > have all this stuff. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From swise at opengridcomputing.com Wed Jul 9 10:45:04 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 12:45:04 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807091938.15779.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> <4874F04B.20603@opengridcomputing.com> <200807091938.15779.olaf.kirch@oracle.com> Message-ID: <4874F920.10102@opengridcomputing.com> Olaf Kirch wrote: > On Wednesday 09 July 2008 19:07:23 Steve Wise wrote: > >> They definitely won't compile in the ofed_1_4 tree as it stands. The >> MEM_MGT_EXTENSIONS patches are still in roland's for-2.6.27 branch. >> > > That's why I included these patches in my test branch :-) > Excellent! Thanks. From swise at opengridcomputing.com Wed Jul 9 11:17:13 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 13:17:13 -0500 Subject: [ofa-general] having problems with the latest for-2.6.27 tree Message-ID: <487500A9.5050707@opengridcomputing.com> Hey Roland, I'm trying out the latest for-2.6.27 tree and I'm seeing hangs doing simple rpings over iwarp. I'm still investigating, but I see this on the server side. Somebody's in a dead lock methinks: INFO: task iw_cm_wq:3042 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. iw_cm_wq D ffffffff8125a5a0 0 3042 2 ffff81002e643d50 0000000000000046 0000000000000000 0000000000000000 ffff81003e51ab20 ffff81003fddd640 ffff81003e51ae70 00000002a039032d ffffffffffffffff ffff810033cb1800 0000000000000000 0000000000000000 Call Trace: [] __mutex_lock_slowpath+0x5c/0x93 [] mutex_lock+0x2b/0x2f [] :rdma_cm:cma_disable_callback+0x19/0x36 [] :rdma_cm:cma_iw_handler+0x1f/0x11c [] :iw_cm:cm_work_handler+0x280/0x3ef [] :iw_cm:cm_work_handler+0x0/0x3ef [] run_workqueue+0x7b/0x103 [] worker_thread+0xd5/0xe0 [] autoremove_wake_function+0x0/0x2e [] worker_thread+0x0/0xe0 [] kthread+0x47/0x76 [] child_rip+0xa/0x12 [] kthread+0x0/0x76 [] child_rip+0x0/0x12 [root at vic9 ~]# From rdreier at cisco.com Wed Jul 9 11:23:09 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 09 Jul 2008 11:23:09 -0700 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: <487500A9.5050707@opengridcomputing.com> (Steve Wise's message of "Wed, 09 Jul 2008 13:17:13 -0500") References: <487500A9.5050707@opengridcomputing.com> Message-ID: maybe a bug in "RDMA/cma: Simplify locking needed for serialization of callbacks"? I'll read it over, or you could try running with lockdep enabled... - R. From swise at opengridcomputing.com Wed Jul 9 11:31:49 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 13:31:49 -0500 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: References: <487500A9.5050707@opengridcomputing.com> Message-ID: <48750415.1020202@opengridcomputing.com> Roland Dreier wrote: > maybe a bug in "RDMA/cma: Simplify locking needed for serialization of > callbacks"? > > I'll read it over, or you could try running with lockdep enabled... > > - R. > I backed that patch out and things work again... From 6bryaustind at deloitte.com Wed Jul 9 09:59:52 2008 From: 6bryaustind at deloitte.com (goran michelle) Date: Wed, 09 Jul 2008 16:59:52 +0000 Subject: [ofa-general] buy 2 products - discount Message-ID: <000901c8e1f4$02dcf8a1$27e42b92@lpujnfwq> " My order arrived yesterday via registered mail in good order THE WATCH IS BEAUTIFUL AND EVEN BETTER THAN I EXPECTED." Try it for yourself - u will be amazed!! - The worlds largest online retailer of luxury products, including: Rolex Sports Models Rolex Datejusts Breitling Cartier Porsche Design Dolce & Gabbana Dior Gucci Hermes Watches Patek Philippe Visit - www.proooe.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Wed Jul 9 11:57:57 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 09 Jul 2008 11:57:57 -0700 Subject: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080709050303.GE12551@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> Message-ID: <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> Sasha, On Wed, 2008-07-09 at 08:03 +0300, Sasha Khapyorsky wrote: > Hi, > > There is a new release of the management (OpenSM and infiniband > diagnostics) tarballs available in: > > http://www.openfabrics.org/downloads/management/ > > md5sum: > > 59737e8ef106c3a37a3bab690397d973 libibcommon-1.1.1.tar.gz > 40d87f68e6259eb25e85f89d27b95f63 libibumad-1.2.1.tar.gz > 891c907cf7fb56191c1cd4224608ef63 libibmad-1.2.1.tar.gz There have been API additions and changes in the libraries since last release. Although the tarball versions have changed, have the ones in the library map files changed ? Don't they need to be updated to reflect the API additions/changes ? -- Hal > fd74b4456987ea78da8f1c2e7edf4f84 opensm-3.2.2.tar.gz > ff809b62b2f6cead7d33fb722d8638b6 infiniband-diags-1.4.1.tar.gz > > All component versions are from recent master branch. Full change log is > below. > > Sasha > > > Al Chu (7): > ids_guid_file manpage entry > add guid_routing_order_file option > rearch __osm_ucast_mgr_process_tbl() usage > implement guid_routing_order_file > fix true/false usage > Fix comment typo > Fix regenerate cache corner case. > > Hal Rosenstock (70): > OpenSM: Add QoS_management_in_OpenSM.txt to opensm/doc directory > OpenSM/doc/modular_routing.txt: Fix typo > OpenSM/doc/QoS_management_in_OpenSM.txt: Remove mention of OFED > management: Support separate SA and SM keys as clarified in IBA 1.2.1 > opensm/osm_sa_mcmember_record.c: Improve log message and some comments relating to SNM > opensm/main.c: Minor change to long option for consolidate_ipv6_snm_req > OpenSM: Add another HP OUI to recognized vendor IDs > opensm/osm_subnet.c: Change comment for IPv6 SNM in options file > OpenSM/osm_sa_mcmember_record.c: Collapse all scopes when consolidating IPv6 SNM > OpenSM release notes: Update to 3.1.11 > opensm/osm_lid_mgr.c: Eliminate some potential NULL pointer dereferences > opensm/osm_lin_fwd_tbl.c: Eliminate potential NULL pointer dereference > OpenSM/osm_sa_mcmember_record.c: Validate some more MGID bits for IPv6 SNM > infiniband-diags/saquery.c: Update for change to osm_mad_pool_init API > opensm/osm_sa_mcmember_record.c: Minor logic change in __get_new_mlid > opensm/osm_ucast_ftree.c: Eliminate unnecessary check in __osm_ftree_sw_tbl_element_create > opensm/osm_pkey.c: Eliminate potential NULL pointer dereference > opensm/osm_port.c: Eliminate potential NULL pointer dereferences > libibmad/fields.c: _set_field64 sets field in network rather than host order > infiniband-diags/mcm_rereg_test.c: Handle error when guid file not found > infiniband-diags/ibsendtrap.c: Support CA and port num > OpenSM/osm_sa_mcmember_record.c: Comment reformatting > OpenSM: Remove some vestigial comments > infiniband-diags/saquery.c: In dump_results, use query_result rather than query_svc_rec > opensm/libvendor/osm_vendor_ibumad_sa.c: Cosmetic changes > OpenSM/osm_sa_portinfo_record.c: Cosmetic comment change > OpenSM/osm_sa_mcmember_record.c: Cosmetic change to error log message > OpenSM/osm_sa_path_record.c: Add some information to some error log messages > opensm/include/iba/ib_types.h: Fix comment > OpenSM/include/opensm: Fix some commentary typos > OpenSM/include/osm_port_profile.h: Fix some typos > OpenSM/libvendor/osm_vendor_ibumad_sa.c: Eliminate unneeded define > opensm/osm_lin_fwd_tbl.c: Minor change to __osm_lin_tbl_compute_obj_size > infiniband-diags/saquery.c: In print_multicast_group_records, only query NodeRecords when needed > opensm/osm_sa_path_record.c: Break up some long OSM_LOG lines > opensm/osm_trap_rcv.c: Break up some long comment lines > opensm/osm_sa_mcmember_record.c: Breakup some long lines > opensm/osm_sa_mcmember_record.c: When consolidating SNM, need separate group per PKey > opensm/man/opensm.8.in: Update consolidate_ipv6_snm_req description > opensm/osm_subnet.c: Fix typo > opensm/osm_sa_mcmember_record.c: Some error message improvements > opensm/include/opensm/osm_multicast.h: Cosmetic changes > opensm/osm_sa_mcmember_record.c: Cosmetic changes > opensm/osm_mcast_mgr.c: Cosmetic comment format change > opensm/osm_lid_mgr.c: Cosmetic formatting changes > opensm/include/osm_port.h: Eliminate some unneeded includes > opensm/include/osm_db.h: Fix some typos > opensm/include/osm_prefix_route.h: Support C++ inclusion > opensm/include/osm_mtree.h: Eliminate unneeded include > libibmad/rpc.c: Better error message in madrpc_init > opensm/osm_port.c: Cosmetic commentary changes > opensm/osm_switch: Cosmetic formatting changes > opensm/osm_subnet.h: Cosmetic formatting changes > opensm/osm_sa_mcmember_record.c: Fix some off by 1 issues related to max_multicast_lid_ho > opensm/osm_sw_info_rcv.c: Cosmetic formatting changes > opensm/osm_lid_mgr.c: Fix off by 1 issue with max_lid > opensm/include/complib/cl_fleximap.h: Improve cl_pfn_fmap_cmp_t PARAMETERS comments > opensm/include/opensm/osm_port_profile.h: Cosmetic formatting changes > opensm/osm_sw_info_rcv.c: Cosmetic formatting change > opensm: Convert mgrp_mlid_tbl into array > opensm/include/opensm/osm_port_profile.h: Cosmetic formatting changes > opensm/include/opensm/osm_port.h: Commentary change > osm_port_profile: Change some variable names for consistency > opensm/man/opensm.8.in: Clarify ignore-guids file format > opensm/osm_mcast_mgr.c: Only send MCG delete notice if not previously done > opensm: Convert unicast LIDs in long messages from hex to decimal > opensm/osm_port_profile: Handle all possible ports > libibmad/src/rpc.c: Eliminate compiler warning > libibmad: Initial changes for congestion management class > infiniband-diags/mcm_rereg_test.c: Cosmetic formatting change > > Ira K. Weiny (2): > libibmad: print warning containing destination port when _do_madrpc fails > libibmad: update portid2str to use inet_ntop and drpath2str functions for gid > > Ira Weiny (11): > opensm/opensm/osm_perfmgr.c: change log level of counter overflow message > opensm/configure.in: Fix the QOS and prefix routes config file default locations > infiniband-diags/scripts/iblinkinfo.pl: fix printing of switch name when port 1 is down. > opensm/opensm/osm_lid_mgr.c: set "send_set" when setting rereg bit > OpenSM: Fix rpm build, /opensm/opensm.conf failed to install > OpenSM: Add a Performance Manager HOWTO to the docs and the dist > infiniband-diags/scripts/ibprint[ca|switch|rt].pl: fix printing by name > infiniband-diags/scripts/ibprint[ca|switch|rt].pl: allow printing of multiple matches but print warning to user that multiple matches were found > infiniband-diags/src/ibnetdiscover.c: Fix the PortInfo data on the port we discover a switch on. > infiniband-diags/src/saquery.c: fix potential core dump > libibmad: add a drpath2str function for easier printing of DR Paths. > > Matthias Blankenhaus (1): > saquery: port selection problem > > Max Matveev (1): > libibmad: allow explicit HCA selection by functions from libibmad > > Ron Livne (1): > libibmad: fix for umad unaligned access of ia64. > > Sasha Khapyorsky (75): > opensm/osm_base.h: fix typo > opensm/osm_lid_mgr.c: cosmetic > opensm/doc/current-routing.txt: add DOR algorithm reference > opensm: merge disable_multicast and no_multicast_option options > opensm: remove unused pfn_ui_* callback options > opensm: port_prof_ignore_file option > opensm: write all OpenSM options to cache file > opensm/scripts/opensm.init.in: fix status command > opensm/scripts: remove not used opensmd template > opensm/scripts: remove opensm.conf usage > opensm: pass file name as parameter to config parser funcs > opensm: config file functions return int > opensm: option to specify config file > opensm: use OSM_DEFAULT_CONFIG_FILE as config file > infiniband-diags/scripts/*.pl: exit 2 on usage errors > infiniband-diags/scripts/*.pl: prevent some zero exists on errors > opensm/osm_lid_mgr.c: indentation fixes > opensm: remove osm_log reference from osm_mad_pool object > opensm: remove some vestigial comments and RCS keywords > saquery: --smkey command line option > opensm: improve log messages in __osm_state_mgr_check_tbl_consistency() > infiniband-diags/man/saquery: add --smkey option to the man page > opensm/configure.in: remove AM_MAINTAINER_MODE > opensm: fix OSM_DEFAULT_SM_KEY byte order > opensm: state default SM_Key value change in usage and man > opensm: replace "(null)" string with null_str variable > infiniband-diags/saquery: split print_multicast_group_records() function > opensm: move remote guids counting to ucast_mgr > opensm: preserve base lid routes > opensm: consolidate config parser logging code > opensm/osm_concole.c: trivial: break comment long lines > opensm/osm_console.c: fix guid parsing > opensm/inculde/*.h: remove empty lines in type definitions > opensm/osm_console: use osm_get_switch_by_guid() > opensm/osm_subnet.h: remove some not used typedefs > osm_vendor_select.h: make OSM_VENDOR_INTF_OPENIB to be default > opensm: rename 'struct _osm_*_t' to 'struct osm_*' > opensm: replace inclusion of osm_vendor.h by osm_vendor_api.h > opensm: make osm_perfmgr_db.* content dependent on PerfMgr availability > opensm/osm_ucast_ftree.c: trivial: remove OSM_LOG_EXIT > opensm/perfmgr: move event_db_dump_file to common OpenSM dump dir > opensm/config: allow space separated strings as values > opensm: move event_plugin from perfmgr to opensm > opensm: support for multiple event plugins > opensm: fix compilation error with --enable-debug > cmoplib/*.h: trivial: remove empty lines in structure definitions > opensm_ucast_mgr: add node name and port number to the logging > opensm/osm_subnet: rename field mgrp_mlid_tbl to mgroups > opensm/osm_sa_mcmember_record: cleanup __get_new_mlid() function > opensm: merge all __get_mgrp_by_mlid() instances > opensm: trivial rename max_unicast_lid_ho and max_multicast_lid_ho > opensm/complib: cl_item_obj macro > opensm/osm_ucast_mgr: some code consolidation and cleanup > opensm/osm_subnet.h: comments formatting > opensm: add --version command line option > opensm/configure.in: fix HAVE_DEFAULT_OPENSM_CONFIG_FILE define generation > opensm: --create-config command line option > opensm/man: add -F. --config option to OpenSM man page > opensm: workaround CL_ASSERT() failure with port_order_list qlist > libibmad: remove intermediate buffer in portid2str() > opensm/osm_log.h: trivial formatting in comments > opensm: minor snprintf() usage improvements > libibcommon: improve snprintf() usage > libibumad: improve snprintf() usage. > opensm: fix snprintf() usage > libibumad: fix snprintf() usage > infiniband-diags: fix snprintf() usage > libibmad: fix snprintf() usage > opensm/osm_sa_lft_record: trivial formatting fixes > opensm: trivial formatting fixes > opensm/osm_log: formatting changes > management: revert configure.in update configure.in on 'make dist' > management/make.dist: daily build updates > management/make.dist: make branch checking unconditional > management: bump management versions > > Tim Meier (1): > opensm: osm_subnet.c - added console loopback mode to opts file & parser > > Vincent Ficet (1): > opensm: fix wrong port_guid initialization > > Yevgeny Kliteynik (7): > opensm/osm_state_mgr.c: fix segmentation fault > infiniband-diags/Makefile.am: fix location of ibdiag_version.h > opensm/ib_types.h: cosmetics - µsec t o usec > opensm/ib_types.h: fixing some wrong comments > opensm/osm_qos_parser.y: TRIVIAL addition in log message > opensm/man/opensm.8.in: TRIVIAL addition in fat-tree description > opensm/osm_state_mgr.c: fixing log message > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From swise at opengridcomputing.com Wed Jul 9 12:04:50 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 14:04:50 -0500 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: <48750415.1020202@opengridcomputing.com> References: <487500A9.5050707@opengridcomputing.com> <48750415.1020202@opengridcomputing.com> Message-ID: <48750BD2.9020800@opengridcomputing.com> Steve Wise wrote: > Roland Dreier wrote: >> maybe a bug in "RDMA/cma: Simplify locking needed for serialization of >> callbacks"? >> >> I'll read it over, or you could try running with lockdep enabled... >> >> - R. >> > > I backed that patch out and things work again... > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general With lock dep stuff on: > ============================================= > [ INFO: possible recursive locking detected ] > 2.6.26-rc7 #3 > --------------------------------------------- > iw_cm_wq/3030 is trying to acquire lock: > (&id_priv->handler_mutex){--..}, at: [] > iw_conn_req_handler+0x76/0x1f6 [rdma_cm] > > but task is already holding lock: > (&id_priv->handler_mutex){--..}, at: [] > cma_disable_callback+0x1b/0x38 [rdma_cm] > > other info that might help us debug this: > 3 locks held by iw_cm_wq/3030: > #0: (iw_cm_wq){--..}, at: [] run_workqueue+0x95/0x1df > #1: (&work->work){--..}, at: [] > run_workqueue+0x95/0x1df > #2: (&id_priv->handler_mutex){--..}, at: [] > cma_disable_callback+0x1b/0x38 [rdma_cm] > > stack backtrace: > Pid: 3030, comm: iw_cm_wq Not tainted 2.6.26-rc7 #3 > > Call Trace: > [] __lock_acquire+0x8fa/0xca0 > [] :rdma_cm:iw_conn_req_handler+0x76/0x1f6 > [] lock_acquire+0x8d/0xb1 > [] :rdma_cm:iw_conn_req_handler+0x76/0x1f6 > [] account+0xd8/0xe8 > [] :rdma_cm:iw_conn_req_handler+0x76/0x1f6 > [] mutex_lock_nested+0xe8/0x260 > [] :rdma_cm:cma_listen_handler+0x0/0x1b > [] :rdma_cm:iw_conn_req_handler+0x76/0x1f6 > [] save_stack_trace+0x22/0x3e > [] save_trace+0x37/0x91 > [] mark_held_locks+0x49/0x67 > [] mark_held_locks+0x49/0x67 > [] kmem_cache_alloc+0x78/0xa2 > [] trace_hardirqs_on+0xfb/0x126 > [] :iw_cm:cm_work_handler+0x0/0x3ef > [] :iw_cm:cm_work_handler+0x0/0x3ef > [] :iw_cm:cm_work_handler+0x15b/0x3ef > [] :iw_cm:cm_work_handler+0x0/0x3ef > [] run_workqueue+0xdf/0x1df > [] worker_thread+0xd8/0xe3 > [] autoremove_wake_function+0x0/0x2e > [] worker_thread+0x0/0xe3 > [] kthread+0x47/0x73 > [] trace_hardirqs_on_thunk+0x35/0x3a > [] child_rip+0xa/0x12 > [] restore_args+0x0/0x30 > [] kthread+0x0/0x73 > [] child_rip+0x0/0x12 From sashak at voltaire.com Wed Jul 9 12:05:38 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 22:05:38 +0300 Subject: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709190538.GO12551@sashak.voltaire.com> On 11:57 Wed 09 Jul , Hal Rosenstock wrote: > > 891c907cf7fb56191c1cd4224608ef63 libibmad-1.2.1.tar.gz > > There have been API additions and changes in the libraries since last > release. Although the tarball versions have changed, have the ones in > the library map files changed ? Don't they need to be updated to reflect > the API additions/changes ? Yes, I missed thiat (thought it was done with the API addition patch). I will update the library version, this will take effect in the next daily build. Sasha From rdreier at cisco.com Wed Jul 9 12:07:58 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 09 Jul 2008 12:07:58 -0700 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: <48750BD2.9020800@opengridcomputing.com> (Steve Wise's message of "Wed, 09 Jul 2008 14:04:50 -0500") References: <487500A9.5050707@opengridcomputing.com> <48750415.1020202@opengridcomputing.com> <48750BD2.9020800@opengridcomputing.com> Message-ID: > With lock dep stuff on: yep, the mutex_lock for conn_id in iw_conn_req_handler should really be mutex_lock_nested(&conn_id->handler_mutex, SINGLE_DEPTH_NESTING); since it's already inside another lock of the same class. but I don't understand why it leads to a real deadlock... they're different mutexes and I don't see how the conn_id lock could already be locked?? - R. From rdreier at cisco.com Wed Jul 9 12:21:54 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 09 Jul 2008 12:21:54 -0700 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: <487500A9.5050707@opengridcomputing.com> (Steve Wise's message of "Wed, 09 Jul 2008 13:17:13 -0500") References: <487500A9.5050707@opengridcomputing.com> Message-ID: It looks like an existing bug that now bites harder after the conversion to mutex... on the normal exit path, iw_conn_req_handler never did cma_enable_remove(conn_id) and now doesn't unlock the mutex. So I think the fix is: diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index ead2b33..ae11d5c 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -1342,8 +1342,11 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, cma_exch(conn_id, CMA_DESTROYING); mutex_unlock(&conn_id->handler_mutex); rdma_destroy_id(&conn_id->id); + goto out; } + mutex_unlock(&conn_id->handler_mutex); + out: if (dev) dev_put(dev); I've rolled this and the mutex_lock_nested() fixes into the patch in my tree and pushed it out... will test rping shortly. Would have been nice for the original author of a patch changing locking to test with lockdep (IB needed the mutex_lock_nested treatment too), but oh well. - R. From rdreier at cisco.com Wed Jul 9 12:24:06 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 09 Jul 2008 12:24:06 -0700 Subject: [ofa-general] Fast Reg Question In-Reply-To: <4874C8B0.4030004@opengridcomputing.com> (Tom Tucker's message of "Wed, 09 Jul 2008 09:18:24 -0500") References: <4874C8B0.4030004@opengridcomputing.com> Message-ID: > I'm a little perplexed by the fast reg WR definition. The context is > that I'm using the Fast Reg verb to map the local memory that is the > data source for an RDMA_WRITE. > > The WR format, however, only takes an rkey. How does this all work > when you're using fast reg to map local memory? Does the WR really > need the mr pointer, or both the lkey and rkey? The IBTA spec seems to > indicate that it needs more information about the MR than just the > rkey. On Mellanox, L_Key and R_Key are always the same, so it doesn't really matter. I think in general the idea would be that the L_Key you have gets updatedd with any "consumer key" changes you make in the WR but otherwise works the same. the WR processing better be able to find the MR by R_Key so I think it's OK. I'll look at the IB spec though. - R. From mpselastic at changemakerweb.co.uk Wed Jul 9 12:58:27 2008 From: mpselastic at changemakerweb.co.uk (Damion Skaggs) Date: Wed, 9 Jul 2008 14:58:27 -0500 Subject: [ofa-general] Don't want to buy unknown them at strange stores? Message-ID: <001701c8e1d4$3e377f80$00d6ab6c@HP1> How make your girlfriend happier. Dont leave her nights unsatisfied. Here! fLauthnone eosinophil fantasy-sf eosphorite erverybody exxpatiate evangelism fake-Billy erformance falkinburg entrapment endpolygon -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Wed Jul 9 13:22:21 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 09 Jul 2008 13:22:21 -0700 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080709190538.GO12551@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> Message-ID: <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-07-09 at 22:05 +0300, Sasha Khapyorsky wrote: > On 11:57 Wed 09 Jul , Hal Rosenstock wrote: > > > 891c907cf7fb56191c1cd4224608ef63 libibmad-1.2.1.tar.gz > > > > There have been API additions and changes in the libraries since last > > release. Although the tarball versions have changed, have the ones in > > the library map files changed ? Don't they need to be updated to reflect > > the API additions/changes ? > > Yes, I missed thiat (thought it was done with the API addition patch). I > will update the library version, Thanks. Is it just libibmad or was it also libibumad as well ? Is it just the *.ver file or also the version in the *.map file ? > this will take effect in the next daily build. The tar balls just issued won't be reissued ? -- Hal > Sasha > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg From hrosenstock at xsigo.com Wed Jul 9 13:46:26 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 09 Jul 2008 13:46:26 -0700 Subject: [ofa-general] Re: [PATCH RFC] opensm/event_plugin: plugin API version 2 In-Reply-To: <20080709040320.GC12551@sashak.voltaire.com> References: <1214501218.13056.620.camel@hrosenstock-ws.xsigo.com> <20080627181931.GO29456@sashak.voltaire.com> <20080627130333.08d783c6.weiny2@llnl.gov> <20080627203745.GA29198@obsidianresearch.com> <20080627230339.GV29456@sashak.voltaire.com> <1214825806.3486.173.camel@hrosenstock-ws.xsigo.com> <20080630142705.GF7229@sashak.voltaire.com> <1214861294.3486.216.camel@hrosenstock-ws.xsigo.com> <20080702131104.GI30305@sashak.voltaire.com> <1215005835.3486.269.camel@hrosenstock-ws.xsigo.com> <20080709040320.GC12551@sashak.voltaire.com> Message-ID: <1215636386.18371.126.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-07-09 at 07:03 +0300, Sasha Khapyorsky wrote: > On 06:37 Wed 02 Jul , Hal Rosenstock wrote: > > > > Because exposing all the data structures, etc. is way more granular (all > > the socalled opaque objects are no longer opaque) than a library API. > > It's a totally different magnitude IMO. > > This is what I said. Perhaps but I sense that we're on opposite ends of the spectrum if we're saying the same thing. > Having "structured" API could lead to an issues > similar to what we saw with libosmcomp. Sure if the API is not supported as stable or only private but there is still a large matter of degree here whereas a structured API is usually more limited than all internals as is being done here. > > > > This is related to what I wrote below about the OpenFabrics licensing > > > > requirements. The idea is if GPL licensing were to be allowed (perhaps > > > > only in some limited context), then there could be two different > > > > packages: dual and GPL. In that way plugins would be more assured of > > > > being compatible with each other and OpenSM. > > > > > > I would prefer to separate packages by its functionality and not due to > > > licensing issues. > > > > Right, licensed based packages do put all related functionality in a > > similar bucket (management) but is that the basis to make such a > > decision ? > > Which decisions? I'm not following. The decision to expose all internals in this manner as well as the decision not to see what license modifications might be possible. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Wed Jul 9 13:46:17 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 9 Jul 2008 23:46:17 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709204617.GR12551@sashak.voltaire.com> On 13:22 Wed 09 Jul , Hal Rosenstock wrote: > > Thanks. Is it just libibmad or was it also libibumad as well ? No API related changes were in libibumad. > Is it > just the *.ver file or also the version in the *.map file ? It is just *.ver file, *.map is symbol versioning - there were additions, no changes in existing functions API. > > this will take effect in the next daily build. > > The tar balls just issued won't be reissued ? There were some commits after this already. Sasha From hrosenstock at xsigo.com Wed Jul 9 13:53:12 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 09 Jul 2008 13:53:12 -0700 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080709204617.GR12551@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> Message-ID: <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-07-09 at 23:46 +0300, Sasha Khapyorsky wrote: > On 13:22 Wed 09 Jul , Hal Rosenstock wrote: > > > > Thanks. Is it just libibmad or was it also libibumad as well ? > > No API related changes were in libibumad. > > Is it > > just the *.ver file or also the version in the *.map file ? > > It is just *.ver file, *.map is symbol versioning - there were additions, > no changes in existing functions API. Isn't that needed too when APIs are added ? Also, some existing functions had minor parameter changes (like const being added). > > > this will take effect in the next daily build. > > > > The tar balls just issued won't be reissued ? > > There were some commits after this already. Shouldn't there be a branch for this if that is an issue ? -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From tom at opengridcomputing.com Wed Jul 9 13:55:26 2008 From: tom at opengridcomputing.com (Tom Tucker) Date: Wed, 09 Jul 2008 15:55:26 -0500 Subject: [ofa-general] Fast Reg Question In-Reply-To: References: <4874C8B0.4030004@opengridcomputing.com> Message-ID: <487525BE.5020903@opengridcomputing.com> Roland Dreier wrote: > > I'm a little perplexed by the fast reg WR definition. The context is > > that I'm using the Fast Reg verb to map the local memory that is the > > data source for an RDMA_WRITE. > > > > The WR format, however, only takes an rkey. How does this all work > > when you're using fast reg to map local memory? Does the WR really > > need the mr pointer, or both the lkey and rkey? The IBTA spec seems to > > indicate that it needs more information about the MR than just the > > rkey. > > On Mellanox, L_Key and R_Key are always the same, Also true for iWARP. > so it doesn't really > matter. I think in general the idea would be that the L_Key you have > gets updatedd with any "consumer key" changes you make in the WR but > otherwise works the same. Fair enough. Use the mr->lkey value in the SGE for subsequent DTO. > the WR processing better be able to find the > MR by R_Key so I think it's OK. > > It just seems a little weird to be supplying the R_Key when you're mapping local memory. > I'll look at the IB spec though. > > The spec refers to a bunch of verification on the L_Key. Obviously, if the L_Key and R_Key are the same the distinction is moot. > - R. > From swise at opengridcomputing.com Wed Jul 9 13:56:02 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 09 Jul 2008 15:56:02 -0500 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: References: <487500A9.5050707@opengridcomputing.com> Message-ID: <487525E2.3050302@opengridcomputing.com> Roland Dreier wrote: > It looks like an existing bug that now bites harder after the conversion > to mutex... on the normal exit path, iw_conn_req_handler never did > cma_enable_remove(conn_id) and now doesn't unlock the mutex. > > So I think the fix is: > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index ead2b33..ae11d5c 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -1342,8 +1342,11 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, > cma_exch(conn_id, CMA_DESTROYING); > mutex_unlock(&conn_id->handler_mutex); > rdma_destroy_id(&conn_id->id); > + goto out; > } > > + mutex_unlock(&conn_id->handler_mutex); > + > out: > if (dev) > dev_put(dev); > > I've rolled this and the mutex_lock_nested() fixes into the patch in my > tree and pushed it out... will test rping shortly. > > Would have been nice for the original author of a patch changing locking > to test with lockdep (IB needed the mutex_lock_nested treatment too), > but oh well. > > - R. > rping works now for me. Thanks, Steve. From sashak at voltaire.com Wed Jul 9 14:17:56 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 00:17:56 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080709211756.GS12551@sashak.voltaire.com> On 13:53 Wed 09 Jul , Hal Rosenstock wrote: > > > > It is just *.ver file, *.map is symbol versioning - there were additions, > > no changes in existing functions API. > > Isn't that needed too when APIs are added ? I don't think. > Also, some existing > functions had minor parameter changes (like const being added). Yes, but it does not touch ABI and doesn't affect a linker. > > > > this will take effect in the next daily build. > > > > > > The tar balls just issued won't be reissued ? > > > > There were some commits after this already. > > Shouldn't there be a branch for this if that is an issue ? And then to do libibmad-1.2.1.1? Looks like an overkill for this particular case - 1.2.* is in development period and the issue is pretty minor - we will fix it in today's daily build and in the next tarball release. Sasha From stan.smith at intel.com Wed Jul 9 14:24:15 2008 From: stan.smith at intel.com (Smith, Stan) Date: Wed, 9 Jul 2008 14:24:15 -0700 Subject: [ofa-general] RE: [ofw] Developer's Workshop at SC'08 In-Reply-To: References: Message-ID: <55CE0347B98FCA468923E5FBC25CB4DC03EAF09C@orsmsx413.amr.corp.intel.com> Woodruff, Robert J wrote: > Hi Guys, > > I have been asked by the OpenFabrics marketing working group to gauge > the possible interest in having a developer workshop during the > week of SC'08 in Austin. There are already plans to have BOFs at SC'08 > to provide customers with updates on the upcoming releases, i.e, OFED > 1.4 > and WinOF 2.0. > > The question is, do you think we need a developers workshop that week > as well > to perhaps plan future work, OFED 1.5, WinOF 2.1 etc. or do you think > a face-to-face meeting is not needed this fall and wait till next > spring at Sonoma > for the next face-to-face. > > Thoughts ? For the Windows Working Group, our bi-weekly teleconferences, and soon to be weekly RC meetings have the communications flowing reasonable well. We are still catching up to Linux w.r.t. functionality. A developers workshop the week of SC'08 would not be a high priority, although others may see benefits. Stan. > > woody > _______________________________________________ > ofw mailing list > ofw at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw From sashak at voltaire.com Wed Jul 9 15:23:15 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:23:15 +0300 Subject: [ofa-general] [PATCH] opensm: improve port_prof_ignore handling Message-ID: <20080709222315.GV12551@sashak.voltaire.com> Instead of keeping special map for switch ports which should be ignored by the link equalization and searching over there at every routing iteration it will keep 'is_prof_ignored' flag as member of osm_physp structure for switch external ports. The parser now uses already common parse_node_map() framework and read the file on each heavy sweep to keep ignore ports list up to date. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_port.h | 5 ++ opensm/include/opensm/osm_port_profile.h | 101 ------------------------------ opensm/include/opensm/osm_subnet.h | 4 - opensm/opensm/main.c | 94 --------------------------- opensm/opensm/osm_subnet.c | 17 ----- opensm/opensm/osm_ucast_mgr.c | 63 +++++++++++++++++-- 6 files changed, 63 insertions(+), 221 deletions(-) diff --git a/opensm/include/opensm/osm_port.h b/opensm/include/opensm/osm_port.h index 3521adc..8fbb28d 100644 --- a/opensm/include/opensm/osm_port.h +++ b/opensm/include/opensm/osm_port.h @@ -110,6 +110,7 @@ typedef struct osm_physp { boolean_t healthy; uint8_t vl_high_limit; unsigned need_update; + unsigned is_prof_ignored; osm_dr_path_t dr_path; osm_pkey_tbl_t pkeys; ib_vl_arb_table_t vl_arb[4]; @@ -152,6 +153,10 @@ typedef struct osm_physp { * When set indicates that port was probably reset and port * related tables (PKey, SL2VL, VLArb) require refreshing. * +* is_prof_ignored +* When set indicates that switch port will be ignored by +* the link load equalization algorithm. +* * dr_path * The directed route path to this port. * diff --git a/opensm/include/opensm/osm_port_profile.h b/opensm/include/opensm/osm_port_profile.h index cb95269..9b33e3a 100644 --- a/opensm/include/opensm/osm_port_profile.h +++ b/opensm/include/opensm/osm_port_profile.h @@ -204,106 +204,5 @@ osm_port_prof_path_count_get(IN const osm_port_profile_t * const p_prof) * SEE ALSO *********/ -/****f* OpenSM: Port Profile Opt/osm_port_prof_is_ignored_port -* NAME -* osm_port_prof_is_ignored_port -* -* DESCRIPTION -* Check to see if this port is to be ignored in path counting. -* This is done by examining the optional list of port_prof_ignore_guids. -* -* SYNOPSIS -*/ -static inline boolean_t -osm_port_prof_is_ignored_port(IN const osm_subn_t * p_subn, - IN ib_net64_t node_guid, IN uint8_t port_num) -{ - const cl_map_t *p_map = &p_subn->port_prof_ignore_guids; - void *p_obj = cl_map_get(p_map, node_guid); - long mask, *addr; - - if (p_obj != NULL) { - /* Test bit corresponding to port_num */ - addr = p_obj; - addr += port_num / (8 * sizeof(long)); - mask = 1L << (port_num % (8 * sizeof(long))); - return ((mask & *addr) != 0); - } - return FALSE; -} -/* -* PARAMETERS -* p_subn -* [in] Pointer to the OSM Subnet object. -* -* node_guid -* [in] The node guid -* -* RETURN VALUE -* Returns TRUE if ignore port mask for requested port number is set. -* FALSE otherwise; -* -* NOTES -* -* SEE ALSO -*********/ - -/****f* OpenSM: Port Profile Opt/osm_port_prof_set_ignored_port -* NAME -* osm_port_prof_set_ignored_port -* -* DESCRIPTION -* Set the ignored property of a port. -* -* SYNOPSIS -*/ -static inline boolean_t -osm_port_prof_set_ignored_port(IN osm_subn_t * p_subn, - IN ib_net64_t node_guid, IN uint8_t port_num) -{ - cl_map_t *p_map = &p_subn->port_prof_ignore_guids; - void *p_obj = cl_map_get(p_map, node_guid); - long mask, *addr; - int insert = 0; - - if (!p_obj) { - p_obj = malloc(sizeof(osm_port_mask_t)); - if (!p_obj) - return FALSE; - memset(p_obj, 0, sizeof(osm_port_mask_t)); - insert = 1; - } - - /* Set bit corresponding to port_num */ - addr = p_obj; - addr += port_num / (8 * sizeof(long)); - mask = 1L << (port_num % (8 * sizeof(long))); - *addr |= mask; - - if (insert) { - if (!cl_map_insert(p_map, node_guid, p_obj)) { - free(p_obj); - return FALSE; - } - } - return TRUE; -} -/* -* PARAMETERS -* p_subn -* [in] Pointer to the OSM Subnet object. -* -* node_guid -* [in] The node guid -* -* RETURN VALUE -* Returns TRUE if the ignore port mask was properly updated. -* FALSE otherwise. -* -* NOTES -* -* SEE ALSO -*********/ - END_C_DECLS #endif /* _OSM_PORT_PROFILE_H_ */ diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index 44f5d55..289684b 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -477,7 +477,6 @@ typedef struct osm_subn { cl_qlist_t sa_sr_list; cl_qlist_t sa_infr_list; cl_ptr_vector_t port_lid_tbl; - cl_map_t port_prof_ignore_guids; ib_net16_t master_sm_base_lid; ib_net16_t sm_base_lid; ib_net64_t sm_port_guid; @@ -532,9 +531,6 @@ typedef struct osm_subn { * Container of pointers to all Port objects in the subent. * Indexed by port LID. * -* port_prof_ignore_guids -* A map of guids to be ignored by port profiling. -* * master_sm_base_lid * The base LID owned by the subnet's master SM. * diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index 0490227..8edce33 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -455,88 +455,6 @@ static ib_net64_t get_port_guid(IN osm_opensm_t * p_osm, uint64_t port_guid) /********************************************************************** **********************************************************************/ -#define OSM_MAX_IGNORE_GUID_LINES_LEN 128 - -static int -parse_ignore_guids_file(IN char *guids_file_name, IN osm_opensm_t * p_osm) -{ - FILE *fh; - char line[OSM_MAX_IGNORE_GUID_LINES_LEN]; - char *p_c, *p_ec; - uint32_t line_num = 0; - uint64_t node_guid; - ib_api_status_t status = IB_SUCCESS; - unsigned int port_num; - - OSM_LOG_ENTER(&p_osm->log); - - fh = fopen(guids_file_name, "r"); - if (fh == NULL) { - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR 0601: " - "Unable to open ignore guids file (%s)\n", - guids_file_name); - status = IB_ERROR; - goto Exit; - } - - /* - * Parse the file and add to the ignore guids map. - */ - while (fgets(line, OSM_MAX_IGNORE_GUID_LINES_LEN, fh) != NULL) { - line_num++; - p_c = line; - while ((*p_c == ' ') && (*p_c != '\0')) - p_c++; - node_guid = strtoull(p_c, &p_ec, 16); - if (p_ec == p_c) { - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR 0602: " - "Error in line (%u): %s\n", line_num, line); - status = IB_ERROR; - goto Exit; - } - - while ((*p_ec == ' ') && (*p_ec != '\0')) - p_ec++; - if (!sscanf(p_ec, "%d", &port_num)) { - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR 0603: " - "Error in line (%u): %s\n", line_num, p_ec); - status = IB_ERROR; - goto Exit; - } - - if (port_num > IB_NODE_NUM_PORTS_MAX) { - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR 0604: " - "Invalid PortNum: 0x%X for Node: 0x%" - PRIx64 "\n", port_num, node_guid); - status = IB_ERROR; - goto Exit; - } - - /* ok insert it */ - if (!osm_port_prof_set_ignored_port(&p_osm->subn, - cl_hton64(node_guid), - port_num)) - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR 0605: " - "osm_port_prof_set_ignored_port failed for " - "Node: 0x%" PRIx64 " PortNum: 0x%X\n", - node_guid, port_num); - else - OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, - "Inserted Node: 0x%" PRIx64 - " PortNum: 0x%X into ignored guids list\n", - node_guid, port_num); - - } - - fclose(fh); - -Exit: - OSM_LOG_EXIT(&p_osm->log); - return (status); -} - -/********************************************************************** - **********************************************************************/ static int daemonize(osm_opensm_t * osm) { @@ -1075,18 +993,6 @@ int main(int argc, char *argv[]) goto Exit; } - /* - * Define some port guids to ignore during path equalization - */ - if (opt.port_prof_ignore_file != NULL) { - status = parse_ignore_guids_file(opt.port_prof_ignore_file, &osm); - if (status != IB_SUCCESS) { - printf("\nError from parse_ignore_guids_file (0x%X)\n", - status); - goto Exit; - } - } - setup_signals(); osm_opensm_sweep(&osm); diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 6a818d4..19351d1 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -100,8 +100,6 @@ void osm_subn_destroy(IN osm_subn_t * const p_subn) osm_prtn_t *p_prtn, *p_next_prtn; osm_mgrp_t *p_mgrp; osm_infr_t *p_infr, *p_next_infr; - cl_map_iterator_t pmask_iter, next_pmask_iter; - osm_port_mask_t *p_port_mask; /* it might be a good idea to de-allocate all known objects */ p_next_node = (osm_node_t *) cl_qmap_head(&p_subn->node_guid_tbl); @@ -161,17 +159,6 @@ void osm_subn_destroy(IN osm_subn_t * const p_subn) cl_ptr_vector_destroy(&p_subn->port_lid_tbl); - next_pmask_iter = cl_map_head(&p_subn->port_prof_ignore_guids); - while (next_pmask_iter != cl_map_end(&p_subn->port_prof_ignore_guids)) { - pmask_iter = next_pmask_iter; - next_pmask_iter = cl_map_next(next_pmask_iter); - p_port_mask = cl_map_obj(pmask_iter); - cl_map_remove_item(&p_subn->port_prof_ignore_guids, pmask_iter); - free(p_port_mask); - } - - cl_map_destroy(&p_subn->port_prof_ignore_guids); - osm_qos_policy_destroy(p_subn->p_qos_policy); while (!cl_is_qlist_empty(&p_subn->prefix_routes_list)) { @@ -213,10 +200,6 @@ osm_subn_init(IN osm_subn_t * const p_subn, p_subn->max_mcast_lid_ho = IB_LID_MCAST_END_HO; p_subn->min_ca_mtu = IB_MAX_MTU; p_subn->min_ca_rate = IB_MAX_RATE; - - /* note that insert and remove are part of the port_profile thing */ - cl_map_init(&p_subn->port_prof_ignore_guids, 10); - p_subn->ignore_existing_lfts = TRUE; /* we assume master by default - so we only need to set it true if STANDBY */ diff --git a/opensm/opensm/osm_ucast_mgr.c b/opensm/opensm/osm_ucast_mgr.c index 9f54486..82d1f4b 100644 --- a/opensm/opensm/osm_ucast_mgr.c +++ b/opensm/opensm/osm_ucast_mgr.c @@ -285,6 +285,8 @@ __osm_ucast_mgr_process_port(IN osm_ucast_mgr_t * const p_mgr, "No path to get to LID %u from switch 0x%" PRIx64 "\n", lid_ho, cl_ntoh64(node_guid)); } else { + osm_physp_t *p = osm_node_get_physp_ptr(p_sw->p_node, port); + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, "Routing LID %u to port 0x%X" " for switch 0x%" PRIx64 "\n", @@ -294,9 +296,7 @@ __osm_ucast_mgr_process_port(IN osm_ucast_mgr_t * const p_mgr, we would like to optionally ignore this port in equalization as in the case of the Mellanox Anafa Internal PCI TCA port */ - is_ignored_by_port_prof = - osm_port_prof_is_ignored_port(p_mgr->p_subn, - node_guid, port); + is_ignored_by_port_prof = p->is_prof_ignored; /* We also would ignore this route if the target lid is of @@ -722,22 +722,75 @@ static void add_port_to_order_list(cl_map_item_t * const p_map_item, void *ctx) port->flag = 0; } +static int mark_ignored_port(void *ctx, uint64_t guid, char *p) +{ + osm_ucast_mgr_t *m = ctx; + osm_node_t *node = osm_get_node_by_guid(m->p_subn, cl_hton64(guid)); + osm_physp_t *physp; + unsigned port; + + if (!node || !node->sw) { + OSM_LOG(m->p_log, OSM_LOG_DEBUG, + "switch with guid 0x%016" PRIx64 " is not found\n", + guid); + return 0; + } + + if (!p || !*p || !(port = strtoul(p, NULL, 0)) || + port >= node->sw->num_ports) { + OSM_LOG(m->p_log, OSM_LOG_DEBUG, + "bad port specified for guid 0x%016" PRIx64 "\n", guid); + return 0; + } + + physp = osm_node_get_physp_ptr(node, port); + if (!physp) + return 0; + + physp->is_prof_ignored = 1; + + return 0; +} + +static void clear_prof_ignore_flag(cl_map_item_t * const p_map_item, void *ctx) +{ + osm_switch_t *sw = (osm_switch_t *)p_map_item; + int i; + + for (i = 1; i < sw->num_ports; i++) { + osm_physp_t *p = osm_node_get_physp_ptr(sw->p_node, i); + if (p) + p->is_prof_ignored = 0; + } +} + static void ucast_mgr_build_lfts(osm_ucast_mgr_t *p_mgr) { cl_qlist_init(&p_mgr->port_order_list); if (p_mgr->p_subn->opt.guid_routing_order_file) { OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Fetching guid routing order file %s\n", + "Fetching guid routing order file \'%s\'\n", p_mgr->p_subn->opt.guid_routing_order_file); if (parse_node_map(p_mgr->p_subn->opt.guid_routing_order_file, add_guid_to_order_list, p_mgr)) OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR : " - "cannot parse guid routing order file %s\n", + "cannot parse guid routing order file \'%s\'\n", p_mgr->p_subn->opt.guid_routing_order_file); } + if (p_mgr->p_subn->opt.port_prof_ignore_file) { + cl_qmap_apply_func(&p_mgr->p_subn->sw_guid_tbl, + clear_prof_ignore_flag, NULL); + if (parse_node_map(p_mgr->p_subn->opt.port_prof_ignore_file, + mark_ignored_port, p_mgr)) { + OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR : " + "cannot parse port prof ignore file \'%s\'\n", + p_mgr->p_subn->opt.port_prof_ignore_file); + } + } + cl_qmap_apply_func(&p_mgr->p_subn->port_guid_tbl, add_port_to_order_list, p_mgr); -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 9 15:25:49 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:25:49 +0300 Subject: [ofa-general] [PATCH] opensm/sa: remove local *get_port_by_guid() wrappers Message-ID: <20080709222549.GW12551@sashak.voltaire.com> Remove local duplicated __osm_*_get_port_by_guid() wrappers, use instead common osm_get_port_by_guid(). Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_sa_lft_record.c | 24 ++---------------------- opensm/opensm/osm_sa_mft_record.c | 20 +------------------- opensm/opensm/osm_sa_sw_info_record.c | 22 +--------------------- 3 files changed, 4 insertions(+), 62 deletions(-) diff --git a/opensm/opensm/osm_sa_lft_record.c b/opensm/opensm/osm_sa_lft_record.c index b67430c..1fcb934 100644 --- a/opensm/opensm/osm_sa_lft_record.c +++ b/opensm/opensm/osm_sa_lft_record.c @@ -113,26 +113,6 @@ Exit: /********************************************************************** **********************************************************************/ -static osm_port_t *__osm_lftr_get_port_by_guid(IN osm_sa_t * sa, - IN uint64_t port_guid) -{ - osm_port_t *p_port; - - CL_PLOCK_ACQUIRE(sa->p_lock); - - p_port = osm_get_port_by_guid(sa->p_subn, port_guid); - if (!p_port) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "ERR 4404: " - "Invalid port GUID 0x%016" PRIx64 "\n", port_guid); - p_port = NULL; - } - - CL_PLOCK_RELEASE(sa->p_lock); - return p_port; -} - -/********************************************************************** - **********************************************************************/ static void __osm_lftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, IN void *context) @@ -150,8 +130,8 @@ __osm_lftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, const osm_physp_t *p_physp; /* In switches, the port guid is the node guid. */ - p_port = - __osm_lftr_get_port_by_guid(sa, p_sw->p_node->node_info.port_guid); + p_port = osm_get_port_by_guid(sa->p_subn, + p_sw->p_node->node_info.port_guid); if (!p_port) { OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4405: " "Failed to find Port by Node Guid:0x%016" PRIx64 diff --git a/opensm/opensm/osm_sa_mft_record.c b/opensm/opensm/osm_sa_mft_record.c index b97e95d..8e04705 100644 --- a/opensm/opensm/osm_sa_mft_record.c +++ b/opensm/opensm/osm_sa_mft_record.c @@ -117,24 +117,6 @@ Exit: /********************************************************************** **********************************************************************/ -static osm_port_t *__osm_mftr_get_port_by_guid(IN osm_sa_t * sa, - IN uint64_t port_guid) -{ - osm_port_t *p_port; - - CL_PLOCK_ACQUIRE(sa->p_lock); - - p_port = osm_get_port_by_guid(sa->p_subn, port_guid); - if (!p_port) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "ERR 4A04: " - "Invalid port GUID 0x%016" PRIx64 "\n", port_guid); - - CL_PLOCK_RELEASE(sa->p_lock); - return p_port; -} - -/********************************************************************** - **********************************************************************/ static void __osm_mftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, IN void *context) @@ -155,7 +137,7 @@ __osm_mftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, /* In switches, the port guid is the node guid. */ p_port = - __osm_mftr_get_port_by_guid(sa, p_sw->p_node->node_info.port_guid); + osm_get_port_by_guid(sa->p_subn, p_sw->p_node->node_info.port_guid); if (!p_port) { OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4A05: " "Failed to find Port by Node Guid:0x%016" PRIx64 diff --git a/opensm/opensm/osm_sa_sw_info_record.c b/opensm/opensm/osm_sa_sw_info_record.c index 3a3667c..9a3c112 100644 --- a/opensm/opensm/osm_sa_sw_info_record.c +++ b/opensm/opensm/osm_sa_sw_info_record.c @@ -105,26 +105,6 @@ Exit: /********************************************************************** **********************************************************************/ -static osm_port_t *__osm_sir_get_port_by_guid(IN osm_sa_t * sa, - IN uint64_t port_guid) -{ - osm_port_t *p_port; - - CL_PLOCK_ACQUIRE(sa->p_lock); - - p_port = osm_get_port_by_guid(sa->p_subn, port_guid); - if (!p_port) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "ERR 5309: " - "Invalid port GUID 0x%016" PRIx64 "\n", port_guid); - p_port = NULL; - } - - CL_PLOCK_RELEASE(sa->p_lock); - return p_port; -} - -/********************************************************************** - **********************************************************************/ static void __osm_sir_rcv_create_sir(IN osm_sa_t * sa, IN const osm_switch_t * const p_sw, @@ -147,7 +127,7 @@ __osm_sir_rcv_create_sir(IN osm_sa_t * sa, /* In switches, the port guid is the node guid. */ p_port = - __osm_sir_get_port_by_guid(sa, p_sw->p_node->node_info.port_guid); + osm_get_port_by_guid(sa->p_subn, p_sw->p_node->node_info.port_guid); if (!p_port) { OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 530A: " "Failed to find Port by Node Guid:0x%016" PRIx64 -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 9 15:28:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:28:08 +0300 Subject: [ofa-general] [PATCH] opensm/osm_sa_lft_record: pass block parameter in host byte order In-Reply-To: <20080709222549.GW12551@sashak.voltaire.com> References: <20080709222549.GW12551@sashak.voltaire.com> Message-ID: <20080709222808.GX12551@sashak.voltaire.com> Pass block parameter in host byte order - eliminate double conversion. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_sa_lft_record.c | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-) diff --git a/opensm/opensm/osm_sa_lft_record.c b/opensm/opensm/osm_sa_lft_record.c index 1fcb934..e69211a 100644 --- a/opensm/opensm/osm_sa_lft_record.c +++ b/opensm/opensm/osm_sa_lft_record.c @@ -73,7 +73,7 @@ static ib_api_status_t __osm_lftr_rcv_new_lftr(IN osm_sa_t * sa, IN const osm_switch_t * const p_sw, IN cl_qlist_t * const p_list, - IN ib_net16_t const lid, IN ib_net16_t const block) + IN ib_net16_t const lid, IN uint16_t const block) { osm_lftr_item_t *p_rec_item; ib_api_status_t status = IB_SUCCESS; @@ -93,7 +93,7 @@ __osm_lftr_rcv_new_lftr(IN osm_sa_t * sa, "New LinearForwardingTable: sw 0x%016" PRIx64 "\n\t\t\t\tblock 0x%02X lid 0x%02X\n", cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), - cl_ntoh16(block), cl_ntoh16(lid)); + block, cl_ntoh16(lid)); memset(p_rec_item, 0, sizeof(*p_rec_item)); @@ -101,8 +101,7 @@ __osm_lftr_rcv_new_lftr(IN osm_sa_t * sa, p_rec_item->rec.block_num = block; /* copy the lft block */ - osm_switch_get_fwd_tbl_block(p_sw, cl_ntoh16(block), - p_rec_item->rec.lft); + osm_switch_get_fwd_tbl_block(p_sw, block, p_rec_item->rec.lft); cl_qlist_insert_tail(p_list, &p_rec_item->list_item); @@ -173,14 +172,13 @@ __osm_lftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, if (min_block > max_block) return; max_block = min_block; - } else /* use as many blocks as "in use" */ + } else /* use as many blocks as "in use" */ min_block = 0; /* so we can add these blocks one by one ... */ for (block = min_block; block <= max_block; block++) __osm_lftr_rcv_new_lftr(sa, p_sw, p_ctxt->p_list, - osm_port_get_base_lid(p_port), - cl_hton16(block)); + osm_port_get_base_lid(p_port), block); } /********************************************************************** -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 9 15:30:30 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:30:30 +0300 Subject: [ofa-general] [PATCH] opensm/osm_log: reverse log level check flow Message-ID: <20080709223030.GY12551@sashak.voltaire.com> Reverse log level flow in osm_log(), just return if requested level does not match. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_log.c | 125 +++++++++++++++++++++++------------------------ 1 files changed, 62 insertions(+), 63 deletions(-) diff --git a/opensm/opensm/osm_log.c b/opensm/opensm/osm_log.c index f9cca98..e1bb647 100644 --- a/opensm/opensm/osm_log.c +++ b/opensm/opensm/osm_log.c @@ -125,80 +125,79 @@ void osm_log(IN osm_log_t * const p_log, #endif /* WIN32 */ /* If this is a call to syslog - always print it */ - if (verbosity & (OSM_LOG_SYS | p_log->level)) { - va_start(args, p_str); - vsprintf(buffer, p_str, args); - va_end(args); - - /* this is a call to the syslog */ - if (verbosity & OSM_LOG_SYS) { - syslog(LOG_INFO, "%s\n", buffer); - - /* SYSLOG should go to stdout too */ - if (p_log->out_port != stdout) { - printf("%s\n", buffer); - fflush(stdout); - } + if (!(verbosity & (OSM_LOG_SYS | p_log->level))) + return; + + va_start(args, p_str); + vsprintf(buffer, p_str, args); + va_end(args); + + /* this is a call to the syslog */ + if (verbosity & OSM_LOG_SYS) { + syslog(LOG_INFO, "%s\n", buffer); + + /* SYSLOG should go to stdout too */ + if (p_log->out_port != stdout) { + printf("%s\n", buffer); + fflush(stdout); + } #ifdef WIN32 - OsmReportState(buffer); + OsmReportState(buffer); #endif /* WIN32 */ - } + } - /* regular log to default out_port */ - cl_spinlock_acquire(&p_log->lock); + /* regular log to default out_port */ + cl_spinlock_acquire(&p_log->lock); - if (p_log->max_size && p_log->count > p_log->max_size) { - /* truncate here */ - fprintf(stderr, - "osm_log: log file exceeds the limit %lu. Truncating.\n", - p_log->max_size); - truncate_log_file(p_log); - } + if (p_log->max_size && p_log->count > p_log->max_size) { + /* truncate here */ + fprintf(stderr, + "osm_log: log file exceeds the limit %lu. Truncating.\n", + p_log->max_size); + truncate_log_file(p_log); + } #ifdef WIN32 - GetLocalTime(&st); - _retry: - ret = - fprintf(p_log->out_port, - "[%02d:%02d:%02d:%03d][%04X] 0x%02x -> %s", - st.wHour, st.wMinute, st.wSecond, st.wMilliseconds, - pid, verbosity, buffer); + GetLocalTime(&st); +_retry: + ret = + fprintf(p_log->out_port, + "[%02d:%02d:%02d:%03d][%04X] 0x%02x -> %s", + st.wHour, st.wMinute, st.wSecond, st.wMilliseconds, + pid, verbosity, buffer); #else - pid = pthread_self(); - _retry: - ret = - fprintf(p_log->out_port, - "%s %02d %02d:%02d:%02d %06d [%04X] 0x%02x -> %s", - (result.tm_mon < - 12 ? month_str[result.tm_mon] : "???"), - result.tm_mday, result.tm_hour, result.tm_min, - result.tm_sec, usecs, pid, verbosity, buffer); + pid = pthread_self(); +_retry: + ret = + fprintf(p_log->out_port, + "%s %02d %02d:%02d:%02d %06d [%04X] 0x%02x -> %s", + (result.tm_mon < + 12 ? month_str[result.tm_mon] : "???"), + result.tm_mday, result.tm_hour, result.tm_min, + result.tm_sec, usecs, pid, verbosity, buffer); #endif - /* flush log */ - if (ret > 0 && - (p_log->flush - || (verbosity & (OSM_LOG_ERROR | OSM_LOG_SYS))) - && fflush(p_log->out_port) < 0) - ret = -1; - - if (ret >= 0) { - log_exit_count = 0; - p_log->count += ret; - } else if (log_exit_count < 3) { - log_exit_count++; - if (errno == ENOSPC && p_log->max_size) { - fprintf(stderr, - "osm_log: write failed: %s. Truncating log file.\n", - strerror(errno)); - truncate_log_file(p_log); - goto _retry; - } - fprintf(stderr, "osm_log: write failed: %s\n", + /* flush log */ + if (ret > 0 && + (p_log->flush || (verbosity & (OSM_LOG_ERROR | OSM_LOG_SYS))) + && fflush(p_log->out_port) < 0) + ret = -1; + + if (ret >= 0) { + log_exit_count = 0; + p_log->count += ret; + } else if (log_exit_count < 3) { + log_exit_count++; + if (errno == ENOSPC && p_log->max_size) { + fprintf(stderr, + "osm_log: write failed: %s. Truncating log file.\n", strerror(errno)); + truncate_log_file(p_log); + goto _retry; } - - cl_spinlock_release(&p_log->lock); + fprintf(stderr, "osm_log: write failed: %s\n", strerror(errno)); } + + cl_spinlock_release(&p_log->lock); } void osm_log_raw(IN osm_log_t * const p_log, -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 9 15:31:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:31:32 +0300 Subject: [ofa-general] [PATCH] opensm/osm_log: osl_log() speedup In-Reply-To: <20080709223030.GY12551@sashak.voltaire.com> References: <20080709223030.GY12551@sashak.voltaire.com> Message-ID: <20080709223132.GZ12551@sashak.voltaire.com> Move all calculations under log level check. This speeds up osm_log() execution time when log level doesn't match in 10-100 times. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_log.c | 10 ++++------ 1 files changed, 4 insertions(+), 6 deletions(-) diff --git a/opensm/opensm/osm_log.c b/opensm/opensm/osm_log.c index e1bb647..d4118b1 100644 --- a/opensm/opensm/osm_log.c +++ b/opensm/opensm/osm_log.c @@ -107,7 +107,6 @@ void osm_log(IN osm_log_t * const p_log, char buffer[LOG_ENTRY_SIZE_MAX]; va_list args; int ret; - #ifdef WIN32 SYSTEMTIME st; uint32_t pid = GetCurrentThreadId(); @@ -117,11 +116,6 @@ void osm_log(IN osm_log_t * const p_log, struct tm result; uint64_t time_usecs; uint32_t usecs; - - time_usecs = cl_get_time_stamp(); - tim = time_usecs / 1000000; - usecs = time_usecs % 1000000; - localtime_r(&tim, &result); #endif /* WIN32 */ /* If this is a call to syslog - always print it */ @@ -165,6 +159,10 @@ _retry: st.wHour, st.wMinute, st.wSecond, st.wMilliseconds, pid, verbosity, buffer); #else + time_usecs = cl_get_time_stamp(); + tim = time_usecs / 1000000; + usecs = time_usecs % 1000000; + localtime_r(&tim, &result); pid = pthread_self(); _retry: ret = -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 9 15:35:34 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:35:34 +0300 Subject: [ofa-general] [PATCH] opensm/OSM_LOG(): wrap osm_log call with log level check In-Reply-To: <20080709223132.GZ12551@sashak.voltaire.com> References: <20080709223030.GY12551@sashak.voltaire.com> <20080709223132.GZ12551@sashak.voltaire.com> Message-ID: <20080709223534.GA12551@sashak.voltaire.com> Even more speed (2-3 times) up for noisy osm_log debug calls - this matters when OpenSM runs with lower log level. Signed-off-by: Sasha Khapyorsky --- And also this means that we can clean up all stuff like: if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) OSM_LOG(sm->p_log, OSM_LOG_DEBUG, "blah-blah-blah\n"); opensm/include/opensm/osm_log.h | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/opensm/include/opensm/osm_log.h b/opensm/include/opensm/osm_log.h index 9f1a17b..741cef4 100644 --- a/opensm/include/opensm/osm_log.h +++ b/opensm/include/opensm/osm_log.h @@ -390,7 +390,10 @@ extern void osm_log_msg_box(osm_log_t *log, osm_log_level_t level, extern void osm_log_raw(IN osm_log_t * const p_log, IN const osm_log_level_t verbosity, IN const char *p_buf); -#define OSM_LOG(log, level, fmt, arg...) osm_log(log, level, "%s: " fmt, __func__, ##arg) +#define OSM_LOG(log, level, fmt, arg...) do { \ + if (osm_log_is_active(log, (level))) \ + osm_log(log, level, "%s: " fmt, __func__, ##arg); \ + } while (0) #define DBG_CL_LOCK 0 -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 9 15:42:03 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 10 Jul 2008 01:42:03 +0300 Subject: [ofa-general] [PATCH] opensm/*/Makefile.am: remove install-exec-hook Message-ID: <20080709224203.GB12551@sashak.voltaire.com> From: Doug Ledford Remove install-exec-hook where additional symbolic links to libraries in form of lib*.so- were created. This patch was triggered by the Fedora package review process. Signed-off-by: Sasha Khapyorsky --- opensm/complib/Makefile.am | 10 ---------- opensm/libvendor/Makefile.am | 11 ----------- opensm/opensm/Makefile.am | 10 ---------- 3 files changed, 0 insertions(+), 31 deletions(-) diff --git a/opensm/complib/Makefile.am b/opensm/complib/Makefile.am index 063bb8b..3f44dbb 100644 --- a/opensm/complib/Makefile.am +++ b/opensm/complib/Makefile.am @@ -79,13 +79,3 @@ libosmcompinclude_HEADERS = $(srcdir)/../include/complib/cl_atomic.h \ # headers are distributed as part of the include dir EXTRA_DIST = $(srcdir)/libosmcomp.map $(srcdir)/libosmcomp.ver - -# as we can not use libtool -release since it actually changes the SONAME -# to the full release name instead of keeping it to the original -# name. What we can do it create a link to the installed file such -# that it will mark it with an alias... -# we find the new lib by traversing the links -install-exec-hook: - if test -L $(DESTDIR)/$(libdir)/libosmcomp-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libosmcomp-$(VERSION).so; fi; \ - lname=`\ls -l $(DESTDIR)/$(libdir)/libosmcomp.so | awk '{print $$NF}'`; \ - ln -s $$lname $(DESTDIR)/$(libdir)/libosmcomp-$(VERSION).so diff --git a/opensm/libvendor/Makefile.am b/opensm/libvendor/Makefile.am index 9fbfc9b..f72dbbe 100644 --- a/opensm/libvendor/Makefile.am +++ b/opensm/libvendor/Makefile.am @@ -88,14 +88,3 @@ libosmvendorinclude_HEADERS = $(HDRS) # headers are distributed as part of the include dir EXTRA_DIST = $(srcdir)/libosmvendor.map $(srcdir)/libosmvendor.ver - -# as we can not use libtool -release since it actually changes the SONAME -# to the full release name instead of keeping it to the original -# name. What we can do it create a link to the installed file such -# that it will mark it with an alias... -# we find the new lib by traversing the links -install-exec-hook: - if test -L $(DESTDIR)/$(libdir)/libosmvendor-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libosmvendor-$(VERSION).so; fi; \ - lname=`\ls -l $(DESTDIR)/$(libdir)/libosmvendor.so | awk '{print $$NF}'`; \ - ln -s $$lname $(DESTDIR)/$(libdir)/libosmvendor_$(with_osmv).so; \ - ln -s $$lname $(DESTDIR)/$(libdir)/libosmvendor-$(VERSION).so diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index acd0b1d..80f5a01 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -119,16 +119,6 @@ EXTRA_DIST = $(srcdir)/libopensm.map $(srcdir)/libopensm.ver \ $(srcdir)/ChangeLog \ $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l -# as we can not use libtool -release since it actually changes the SONAME -# to the full release name instead of keeping it to the original -# name. What we can do it create a link to the installed file such -# that it will mark it with an alias... -# we find the new lib by traversing the links -install-exec-hook: - if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ - lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ - ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so - # generate c and h files from the lex and yacc files dist-hook: $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c -- 1.5.5.1.178.g1f811 From jyffd at bradley-homes.com Thu Jul 10 00:38:31 2008 From: jyffd at bradley-homes.com (jyffd at bradley-homes.com) Date: Thu, 10 Jul 2008 07:38:31 -0000 Subject: [ofa-general] Er. Ect harder! Message-ID: <4ba29745$3da37b47$6addd5bb@jyffd> Er. Ect harder! And Satisfy! http://kabirso.cn/a/ From jackm at dev.mellanox.co.il Thu Jul 10 00:39:41 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 10:39:41 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] =?iso-8859-1?q?ib/ipoib=3A=09blocking_multicast_loopback_ipoib?= packets In-Reply-To: <48736C99.4090308@voltaire.com> References: <48736C99.4090308@voltaire.com> Message-ID: <200807101039.41832.jackm@dev.mellanox.co.il> On Tuesday 08 July 2008 16:33, Or Gerlitz wrote: > Assuming this solution (a create-qp-extended verb that gets also > creation flags) would be the one implemented at libibverbs, are you fine > with applying the same approach to librdmacm that is adding a matching > call eg rdma-create-qp-extended that just calls the new verb? Please use "expanded", not "extended", to avoid any confusion with XRC (Extended RC) qp's. - Jack From ronli.voltaire at gmail.com Thu Jul 10 00:47:23 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Thu, 10 Jul 2008 10:47:23 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <200807101039.41832.jackm@dev.mellanox.co.il> References: <48736C99.4090308@voltaire.com> <200807101039.41832.jackm@dev.mellanox.co.il> Message-ID: <3b5e77ad0807100047u1790dd7chf5b59f14be0684a4@mail.gmail.com> No problem, expanded it is. >can we do this so that the new verb lies after XRC in ib_context, since OFED 1.3 went out >with XRC in userspace (so we don't break binary compatibility): > >struct ibv_context { > struct ibv_device *device; > struct ibv_context_ops ops; > int cmd_fd; > int async_fd; > int num_comp_vectors; > pthread_mutex_t mutex; > void *abi_compat; > struct ibv_xrc_ops *xrc_ops; > /* ==> new verb ops structure pointer here */ >}; I'm going to release some patches, based on your XRC patches, in which I'm going to replace xrc_ops with more_ops. These patches will will be relative to your patches. I hope it's ok with you. Ron On Thu, Jul 10, 2008 at 10:39 AM, Jack Morgenstein wrote: > On Tuesday 08 July 2008 16:33, Or Gerlitz wrote: >> Assuming this solution (a create-qp-extended verb that gets also >> creation flags) would be the one implemented at libibverbs, are you fine >> with applying the same approach to librdmacm that is adding a matching >> call eg rdma-create-qp-extended that just calls the new verb? > > Please use "expanded", not "extended", to avoid any confusion with > XRC (Extended RC) qp's. > > - Jack > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From amirv at mellanox.co.il Thu Jul 10 00:48:50 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 10 Jul 2008 10:48:50 +0300 Subject: [ofa-general] [PATCH v2 3/6] Split sdp_handle_wc() to smaller functions In-Reply-To: <12156761332878-git-send-email-amirv@mellanox.co.il> References: <12156761331725-git-send-email-amirv@mellanox.co.il> <12156761332878-git-send-email-amirv@mellanox.co.il> Message-ID: <12156761333757-git-send-email-amirv@mellanox.co.il> Signed-off-by: Amir Vadai --- drivers/infiniband/ulp/sdp/sdp_bcopy.c | 209 ++++++++++++++++++-------------- 1 files changed, 116 insertions(+), 93 deletions(-) diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c index 05825bf..3a8d5ac 100644 --- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c +++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c @@ -588,112 +588,135 @@ static void sdp_handle_resize_ack(struct sdp_sock *ssk, struct sdp_chrecvbuf *bu ssk->sent_request = 0; } -static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) +static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) { + int frags; struct sk_buff *skb; struct sdp_bsdh *h; int pagesz, i; - if (wc->wr_id & SDP_OP_RECV) { - skb = sdp_recv_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; + skb = sdp_recv_completion(ssk, wc->wr_id); + if (unlikely(!skb)) + return -1; - atomic_sub(SDP_MAX_SEND_SKB_FRAGS, &sdp_current_mem_usage); + atomic_sub(SDP_MAX_SEND_SKB_FRAGS, &sdp_current_mem_usage); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, - "Recv completion with error. " - "Status %d\n", wc->status); - sdp_reset(&ssk->isk.sk); - } - __kfree_skb(skb); - } else { - int frags; - - sdp_dbg_data(&ssk->isk.sk, - "Recv completion. ID %d Length %d\n", - (int)wc->wr_id, wc->byte_len); - if (unlikely(wc->byte_len < sizeof(struct sdp_bsdh))) { - printk("SDP BUG! byte_len %d < %zd\n", - wc->byte_len, sizeof(struct sdp_bsdh)); - __kfree_skb(skb); - return; - } - skb->len = wc->byte_len; - if (likely(wc->byte_len > SDP_HEAD_SIZE)) - skb->data_len = wc->byte_len - SDP_HEAD_SIZE; - else - skb->data_len = 0; - skb->data = skb->head; + if (unlikely(wc->status)) { + if (wc->status != IB_WC_WR_FLUSH_ERR) { + sdp_dbg(&ssk->isk.sk, + "Recv completion with error. " + "Status %d\n", wc->status); + sdp_reset(&ssk->isk.sk); + } + __kfree_skb(skb); + return 0; + } + + sdp_dbg_data(&ssk->isk.sk, "Recv completion. ID %d Length %d\n", + (int)wc->wr_id, wc->byte_len); + if (unlikely(wc->byte_len < sizeof(struct sdp_bsdh))) { + printk(KERN_WARNING "SDP BUG! byte_len %d < %zd\n", + wc->byte_len, sizeof(struct sdp_bsdh)); + __kfree_skb(skb); + return -1; + } + skb->len = wc->byte_len; + if (likely(wc->byte_len > SDP_HEAD_SIZE)) + skb->data_len = wc->byte_len - SDP_HEAD_SIZE; + else + skb->data_len = 0; + skb->data = skb->head; #ifdef NET_SKBUFF_DATA_USES_OFFSET - skb->tail = skb_headlen(skb); + skb->tail = skb_headlen(skb); #else - skb->tail = skb->head + skb_headlen(skb); + skb->tail = skb->head + skb_headlen(skb); #endif - h = (struct sdp_bsdh *)skb->data; - skb_reset_transport_header(skb); - ssk->mseq_ack = ntohl(h->mseq); - if (ssk->mseq_ack != (int)wc->wr_id) - printk("SDP BUG! mseq %d != wrid %d\n", - ssk->mseq_ack, (int)wc->wr_id); - ssk->bufs = ntohl(h->mseq_ack) - ssk->tx_head + 1 + - ntohs(h->bufs); - - frags = skb_shinfo(skb)->nr_frags; - pagesz = PAGE_ALIGN(skb->data_len); - skb_shinfo(skb)->nr_frags = pagesz / PAGE_SIZE; - - for (i = skb_shinfo(skb)->nr_frags; - i < frags; ++i) { - put_page(skb_shinfo(skb)->frags[i].page); - skb->truesize -= PAGE_SIZE; - } - - if (unlikely(h->flags & SDP_OOB_PEND)) - sk_send_sigurg(&ssk->isk.sk); - - skb_pull(skb, sizeof(struct sdp_bsdh)); - - if (likely(h->mid == SDP_MID_DATA) && - likely(skb->len > 0)) { - int oob = h->flags & SDP_OOB_PRES; - skb = sdp_sock_queue_rcv_skb(&ssk->isk.sk, skb); - if (unlikely(oob)) - sdp_urg(ssk, skb); - } else if (likely(h->mid == SDP_MID_DATA)) { - __kfree_skb(skb); - } else if (h->mid == SDP_MID_DISCONN) { - /* this will wake recvmsg */ - sdp_sock_queue_rcv_skb(&ssk->isk.sk, skb); - sdp_fin(&ssk->isk.sk); - } else if (h->mid == SDP_MID_CHRCVBUF) { - sdp_handle_resize_request(ssk, (struct sdp_chrecvbuf *)skb->data); - __kfree_skb(skb); - } else if (h->mid == SDP_MID_CHRCVBUF_ACK) { - sdp_handle_resize_ack(ssk, (struct sdp_chrecvbuf *)skb->data); - __kfree_skb(skb); - } else { - /* TODO: Handle other messages */ - printk("SDP: FIXME MID %d\n", h->mid); - __kfree_skb(skb); - } + h = (struct sdp_bsdh *)skb->data; + skb_reset_transport_header(skb); + ssk->mseq_ack = ntohl(h->mseq); + if (ssk->mseq_ack != (int)wc->wr_id) + printk(KERN_WARNING "SDP BUG! mseq %d != wrid %d\n", + ssk->mseq_ack, (int)wc->wr_id); + ssk->bufs = ntohl(h->mseq_ack) - ssk->tx_head + 1 + + ntohs(h->bufs); + + frags = skb_shinfo(skb)->nr_frags; + pagesz = PAGE_ALIGN(skb->data_len); + skb_shinfo(skb)->nr_frags = pagesz / PAGE_SIZE; + + for (i = skb_shinfo(skb)->nr_frags; + i < frags; ++i) { + put_page(skb_shinfo(skb)->frags[i].page); + skb->truesize -= PAGE_SIZE; + } + + if (unlikely(h->flags & SDP_OOB_PEND)) + sk_send_sigurg(&ssk->isk.sk); + + skb_pull(skb, sizeof(struct sdp_bsdh)); + + switch (h->mid) { + case SDP_MID_DATA: + if (unlikely(skb->len <= 0)) { + __kfree_skb(skb); + break; } + skb = sdp_sock_queue_rcv_skb(&ssk->isk.sk, skb); + if (unlikely(h->flags & SDP_OOB_PRES)) + sdp_urg(ssk, skb); + break; + case SDP_MID_DISCONN: + /* this will wake recvmsg */ + sdp_sock_queue_rcv_skb(&ssk->isk.sk, skb); + sdp_fin(&ssk->isk.sk); + break; + case SDP_MID_CHRCVBUF: + sdp_handle_resize_request(ssk, + (struct sdp_chrecvbuf *)skb->data); + __kfree_skb(skb); + break; + + case SDP_MID_CHRCVBUF_ACK: + sdp_handle_resize_ack(ssk, (struct sdp_chrecvbuf *)skb->data); + __kfree_skb(skb); + break; + default: + /* TODO: Handle other messages */ + printk(KERN_WARNING "SDP: FIXME MID %d\n", h->mid); + __kfree_skb(skb); + } + return 0; +} + +static int sdp_handle_send_comp(struct sdp_sock *ssk, struct ib_wc *wc) +{ + struct sk_buff *skb; + + skb = sdp_send_completion(ssk, wc->wr_id); + if (unlikely(!skb)) + return -1; + sk_wmem_free_skb(&ssk->isk.sk, skb); + if (unlikely(wc->status)) { + if (wc->status != IB_WC_WR_FLUSH_ERR) { + sdp_dbg(&ssk->isk.sk, + "Send completion with error. " + "Status %d\n", wc->status); + sdp_set_error(&ssk->isk.sk, -ECONNRESET); + wake_up(&ssk->wq); + } + } + + return 0; +} + +static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) +{ + if (wc->wr_id & SDP_OP_RECV) { + if (sdp_handle_recv_comp(ssk, wc)) + return; } else if (likely(wc->wr_id & SDP_OP_SEND)) { - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) + if (sdp_handle_send_comp(ssk, wc)) return; - sk_wmem_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, - "Send completion with error. " - "Status %d\n", wc->status); - sdp_set_error(&ssk->isk.sk, -ECONNRESET); - wake_up(&ssk->wq); - } - } } else { sdp_cnt(sdp_keepalive_probes_sent); -- 1.5.3 From amirv at mellanox.co.il Thu Jul 10 00:48:48 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 10 Jul 2008 10:48:48 +0300 Subject: [ofa-general] [PATCH v2 1/6] Use sdp_set_state() Message-ID: <12156761331725-git-send-email-amirv@mellanox.co.il> Fix all places that change sk_state directly to use sdp_set_state() Signed-off-by: Amir Vadai --- drivers/infiniband/ulp/sdp/sdp.h | 12 ++++++------ drivers/infiniband/ulp/sdp/sdp_bcopy.c | 4 ++-- drivers/infiniband/ulp/sdp/sdp_cma.c | 6 +++--- drivers/infiniband/ulp/sdp/sdp_main.c | 18 +++++++++--------- 4 files changed, 20 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h index bb2434a..5bd4041 100644 --- a/drivers/infiniband/ulp/sdp/sdp.h +++ b/drivers/infiniband/ulp/sdp/sdp.h @@ -218,13 +218,18 @@ static inline struct sdp_sock *sdp_sk(const struct sock *sk) return (struct sdp_sock *)sk; } +static inline void sdp_set_state(struct sock *sk, int state) +{ + sk->sk_state = state; +} + static inline void sdp_set_error(struct sock *sk, int err) { sk->sk_err = -err; if (sk->sk_socket) sk->sk_socket->state = SS_UNCONNECTED; - sk->sk_state = TCP_CLOSE; + sdp_set_state(sk, TCP_CLOSE); if (sdp_sk(sk)->time_wait) { sdp_dbg(sk, "%s: destroy in time wait state\n", __func__); @@ -235,11 +240,6 @@ static inline void sdp_set_error(struct sock *sk, int err) sk->sk_error_report(sk); } -static inline void sdp_set_state(struct sock *sk, int state) -{ - sk->sk_state = state; -} - extern struct workqueue_struct *sdp_workqueue; int sdp_cma_handler(struct rdma_cm_id *, struct rdma_cm_event *); diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c index be7c598..05825bf 100644 --- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c +++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c @@ -544,9 +544,9 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) BUG_ON(!skb); sdp_post_send(ssk, skb, SDP_MID_DISCONN); if (ssk->isk.sk.sk_state == TCP_FIN_WAIT1) - ssk->isk.sk.sk_state = TCP_FIN_WAIT2; + sdp_set_state(&ssk->isk.sk, TCP_FIN_WAIT2); else - ssk->isk.sk.sk_state = TCP_CLOSING; + sdp_set_state(&ssk->isk.sk, TCP_CLOSING); } } diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c index 2e4ceb5..f2fb083 100644 --- a/drivers/infiniband/ulp/sdp/sdp_cma.c +++ b/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -256,7 +256,7 @@ int sdp_connect_handler(struct sock *sk, struct rdma_cm_id *id, list_add_tail(&sdp_sk(child)->backlog_queue, &sdp_sk(sk)->backlog_queue); sdp_sk(child)->parent = sk; - child->sk_state = TCP_SYN_RECV; + sdp_set_state(child, TCP_SYN_RECV); /* child->sk_write_space(child); */ /* child->sk_data_ready(child, 0); */ @@ -272,7 +272,7 @@ static int sdp_response_handler(struct sock *sk, struct rdma_cm_id *id, struct sockaddr_in *dst_addr; sdp_dbg(sk, "%s\n", __func__); - sk->sk_state = TCP_ESTABLISHED; + sdp_set_state(sk, TCP_ESTABLISHED); if (sock_flag(sk, SOCK_KEEPOPEN)) sdp_start_keepalive_timer(sk); @@ -316,7 +316,7 @@ int sdp_connected_handler(struct sock *sk, struct rdma_cm_event *event) parent = sdp_sk(sk)->parent; BUG_ON(!parent); - sk->sk_state = TCP_ESTABLISHED; + sdp_set_state(sk, TCP_ESTABLISHED); if (sock_flag(sk, SOCK_KEEPOPEN)) sdp_start_keepalive_timer(sk); diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c index 8f1e8b0..acd3aab 100644 --- a/drivers/infiniband/ulp/sdp/sdp_main.c +++ b/drivers/infiniband/ulp/sdp/sdp_main.c @@ -452,9 +452,9 @@ static int sdp_close_state(struct sock *sk) return 0; if (sk->sk_state == TCP_ESTABLISHED) - sk->sk_state = TCP_FIN_WAIT1; + sdp_set_state(sk, TCP_FIN_WAIT1); else if (sk->sk_state == TCP_CLOSE_WAIT) - sk->sk_state = TCP_LAST_ACK; + sdp_set_state(sk, TCP_LAST_ACK); else return 0; return 1; @@ -553,7 +553,7 @@ adjudge_to_death: if (sk->sk_state == TCP_FIN_WAIT2 && !sk->sk_send_head && sdp_sk(sk)->tx_head == sdp_sk(sk)->tx_tail) { - sk->sk_state = TCP_CLOSE; + sdp_set_state(sk, TCP_CLOSE); } if ((1 << sk->sk_state) & (TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2)) { @@ -622,7 +622,7 @@ static int sdp_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) return rc; } - sk->sk_state = TCP_SYN_SENT; + sdp_set_state(sk, TCP_SYN_SENT); return 0; } @@ -853,7 +853,7 @@ void sdp_time_wait_work(struct work_struct *work) sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - sk->sk_state = TCP_CLOSE; + sdp_set_state(sk, TCP_CLOSE); sdp_sk(sk)->time_wait = 0; release_sock(sk); @@ -864,7 +864,7 @@ void sdp_time_wait_work(struct work_struct *work) void sdp_time_wait_destroy_sk(struct sdp_sock *ssk) { ssk->time_wait = 0; - ssk->isk.sk.sk_state = TCP_CLOSE; + sdp_set_state(&ssk->isk.sk, TCP_CLOSE); queue_work(sdp_workqueue, &ssk->destroy_work); } @@ -895,9 +895,9 @@ static void sdp_shutdown(struct sock *sk, int how) return; if (sk->sk_state == TCP_ESTABLISHED) - sk->sk_state = TCP_FIN_WAIT1; + sdp_set_state(sk, TCP_FIN_WAIT1); else if (sk->sk_state == TCP_CLOSE_WAIT) - sk->sk_state = TCP_LAST_ACK; + sdp_set_state(sk, TCP_LAST_ACK); else return; @@ -1991,7 +1991,7 @@ static int sdp_listen(struct sock *sk, int backlog) sdp_warn(sk, "rdma_listen failed: %d\n", rc); sdp_set_error(sk, rc); } else - sk->sk_state = TCP_LISTEN; + sdp_set_state(sk, TCP_LISTEN); return rc; } -- 1.5.3 From amirv at mellanox.co.il Thu Jul 10 00:48:53 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 10 Jul 2008 10:48:53 +0300 Subject: [ofa-general] [PATCH v2 6/6] Fixed reference accounting In-Reply-To: <12156761333073-git-send-email-amirv@mellanox.co.il> References: <12156761331725-git-send-email-amirv@mellanox.co.il> <12156761332878-git-send-email-amirv@mellanox.co.il> <12156761333757-git-send-email-amirv@mellanox.co.il> <12156761331845-git-send-email-amirv@mellanox.co.il> <12156761333073-git-send-email-amirv@mellanox.co.il> Message-ID: <12156761331111-git-send-email-amirv@mellanox.co.il> Signed-off-by: Amir Vadai --- drivers/infiniband/ulp/sdp/sdp.h | 63 +++++++++++- drivers/infiniband/ulp/sdp/sdp_bcopy.c | 56 ++++-------- drivers/infiniband/ulp/sdp/sdp_cma.c | 22 ++++- drivers/infiniband/ulp/sdp/sdp_main.c | 98 ++++++++++---------- .../2.6.16/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.17/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.18/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.19/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.20/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.21/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.22/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.23/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.24/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../2.6.9_U4/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../backport/2.6.9_U4/sdp_7277_to_2_6_11.patch | 18 ++-- .../2.6.9_U5/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../backport/2.6.9_U5/sdp_7277_to_2_6_11.patch | 18 ++-- .../2.6.9_U6/sdp_0090_revert_to_2_6_24.patch | 72 ++++++++------- .../backport/2.6.9_U6/sdp_7277_to_2_6_11.patch | 18 ++-- 27 files changed, 928 insertions(+), 805 deletions(-) diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h index 3b2dffb..ffe2182 100644 --- a/drivers/infiniband/ulp/sdp/sdp.h +++ b/drivers/infiniband/ulp/sdp/sdp.h @@ -110,7 +110,7 @@ struct sdp_sock { struct work_struct work; wait_queue_head_t wq; - struct delayed_work time_wait_work; + struct delayed_work fin_wait_work; struct work_struct destroy_work; /* Like tcp_sock */ @@ -129,10 +129,13 @@ struct sdp_sock { unsigned keepalive_time; + spinlock_t lock; + /* tx_head/rx_head when keepalive timer started */ unsigned keepalive_tx_head; unsigned keepalive_rx_head; + int destructed_already; int sdp_disconnect; /* Data below will be reset on error */ @@ -220,21 +223,69 @@ static inline struct sdp_sock *sdp_sk(const struct sock *sk) return (struct sdp_sock *)sk; } -static inline void sdp_set_state(struct sock *sk, int state) +static inline char *sdp_state_str(int state) { + static char *state_str[] = { + [TCP_ESTABLISHED] = "TCP_ESTABLISHED", + [TCP_SYN_SENT] = "TCP_SYN_SENT", + [TCP_SYN_RECV] = "TCP_SYN_RECV", + [TCP_FIN_WAIT1] = "TCP_FIN_WAIT1", + [TCP_FIN_WAIT2] = "TCP_FIN_WAIT2", + [TCP_TIME_WAIT] = "TCP_TIME_WAIT", + [TCP_CLOSE] = "TCP_CLOSE", + [TCP_CLOSE_WAIT] = "TCP_CLOSE_WAIT", + [TCP_LAST_ACK] = "TCP_LAST_ACK", + [TCP_LISTEN] = "TCP_LISTEN", + [TCP_CLOSING] = "TCP_CLOSING", + }; + + if (state < 0 || state >= TCP_MAX_STATES) + return "unknown"; + + return state_str[state]; +} + +static inline int _sdp_exch_state(const char *func, int line, struct sock *sk, + int state) +{ + unsigned long flags; + int old; + + sdp_dbg(sk, "%s:%d - set state: %s -> %s 0x%x\n", __func__, __LINE__, + sdp_state_str(old_state), sdp_state_str(state), orig_states); + + if ((1 << old_state) & ~(orig_states)) { + sdp_warn(sk, "trying to exchange state from unexpected state " + "%s to state %s. expected states: 0x%x\n", + sdp_state_str(old_state), sdp_state_str(state), + orig_states); + dump_stack(); + } + + spin_lock_irqsave(&sdp_sk(sk)->lock, flags); + + old = sk->sk_state; sk->sk_state = state; + + spin_unlock_irqrestore(&sdp_sk(sk)->lock, flags); + + return old; } +#define sdp_exch_state(sk, orig_states, state) \ + _sdp_exch_state(__func__, __LINE__, sk, state); static inline void sdp_set_error(struct sock *sk, int err) { + unsigned char ib_teardown_states = TCPF_FIN_WAIT1 | TCPF_CLOSE_WAIT + | TCPF_LAST_ACK; sk->sk_err = -err; if (sk->sk_socket) sk->sk_socket->state = SS_DISCONNECTING; - if (sk->sk_state == TCP_SYN_SENT) - sdp_set_state(sk, TCP_CLOSE); + if ((1 << sk->sk_state) & ib_teardown_states) + sdp_exch_state(sk, ib_teardown_states); else - sdp_set_state(sk, TCP_TIME_WAIT); + sdp_exch_state(sk, ~0, TCP_CLOSE); sk->sk_error_report(sk); } @@ -254,7 +305,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle); void sdp_destroy_work(struct work_struct *work); void sdp_cancel_fin_wait_timeout(struct sdp_sock *ssk); void sdp_fin_work(struct work_struct *work); -void sdp_time_wait_work(struct work_struct *work); +void sdp_fin_wait_work(struct work_struct *work); struct sk_buff *sdp_recv_completion(struct sdp_sock *ssk, int id); struct sk_buff *sdp_send_completion(struct sdp_sock *ssk, int mseq); void sdp_urg(struct sdp_sock *ssk, struct sk_buff *skb); diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c index d3f50b5..77f2d99 100644 --- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c +++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c @@ -107,22 +107,15 @@ static void sdp_fin(struct sock *sk) switch (sk->sk_state) { case TCP_SYN_RECV: case TCP_ESTABLISHED: - sdp_set_state(sk, TCP_CLOSE_WAIT); + sdp_exch_state(sk, TCPF_SYN_RECV | TCPF_ESTABLISHED, + TCP_CLOSE_WAIT); break; case TCP_FIN_WAIT1: - /* This case occurs when a simultaneous close - * happens, we must ack the received FIN and - * enter the CLOSING state. - */ - sdp_set_state(sk, TCP_CLOSING); - break; - - case TCP_FIN_WAIT2: /* Received a reply FIN - start Infiniband tear down */ - sdp_set_state(sk, TCP_TIME_WAIT); sdp_dbg(sk, "%s: Starting Infiniband tear down sending DREQ\n", __func__); + sdp_exch_state(sk, TCPF_FIN_WAIT1, TCP_TIME_WAIT); if (sdp_sk(sk)->id) { rdma_disconnect(sdp_sk(sk)->id); @@ -132,7 +125,8 @@ static void sdp_fin(struct sock *sk) } break; case TCP_TIME_WAIT: - case TCP_CLOSE: + /* This is a mutual close situation and we've got the DREQ from + the peer before the SDP_MID_DISCONNECT */ break; default: sdp_warn(sk, "%s: FIN in unexpected state. sk->sk_state=%d\n", @@ -425,7 +419,7 @@ static inline struct sk_buff *sdp_sock_queue_rcv_skb(struct sock *sk, { int skb_len; struct sdp_sock *ssk = sdp_sk(sk); - struct sk_buff *tail; + struct sk_buff *tail = NULL; /* not needed since sk_rmem_alloc is not currently used * TODO - remove this? @@ -562,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { skb = sdp_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); @@ -698,10 +692,10 @@ static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) if (unlikely(sk->sk_shutdown & RCV_SHUTDOWN)) { /* got data in RCV_SHUTDOWN */ - if ((1 << sk->sk_state) & - (TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2)) { + if (sk->sk_state == TCP_FIN_WAIT1) { /* go into abortive close */ - sdp_set_state(sk, TCP_TIME_WAIT); + sdp_exch_state(sk, TCPF_FIN_WAIT1, + TCP_TIME_WAIT); sk->sk_prot->disconnect(sk, 0); } @@ -749,10 +743,13 @@ static int sdp_handle_send_comp(struct sdp_sock *ssk, struct ib_wc *wc) if (unlikely(wc->status)) { if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, "Send completion with error. " - "Status %d\n", wc->status); - sdp_set_error(&ssk->isk.sk, -ECONNRESET); + struct sock *sk = &ssk->isk.sk; + sdp_dbg(sk, "Send completion with error. " + "Status %d\n", wc->status); + sdp_set_error(sk, -ECONNRESET); wake_up(&ssk->wq); + + queue_work(sdp_workqueue, &ssk->destroy_work); } goto out; } @@ -762,26 +759,7 @@ static int sdp_handle_send_comp(struct sdp_sock *ssk, struct ib_wc *wc) if (likely(h->mid != SDP_MID_DISCONN)) goto out; - switch (ssk->isk.sk.sk_state) { - case TCP_FIN_WAIT1: - /* sdp_set_state(&ssk->isk.sk, - TCP_FIN_WAIT2); */ - break; - case TCP_CLOSING: - case TCP_LAST_ACK: - sdp_set_state(&ssk->isk.sk, TCP_TIME_WAIT); - - sdp_dbg(&ssk->isk.sk, - "%s: waiting for Infiniband tear down\n", __func__); - - /* We will not issue a DREQ here - because DREQ is sent on - the CM QP - and could reach the peer before the - SDP_MID_DISCONNECT This will put the peer in abortive close - state - without any real reason. - TODO: maybe a timeout whould be used here - what if the - peer won't send a DREQ? */ - break; - default: + if ((1 << ssk->isk.sk.sk_state) & ~(TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) { sdp_dbg(&ssk->isk.sk, "%s: sent DISCONNECT from unexpected state %d\n", __func__, ssk->isk.sk.sk_state); diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c index 8759bac..768c862 100644 --- a/drivers/infiniband/ulp/sdp/sdp_cma.c +++ b/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -220,7 +220,7 @@ int sdp_connect_handler(struct sock *sk, struct rdma_cm_id *id, sdp_add_sock(sdp_sk(child)); INIT_LIST_HEAD(&sdp_sk(child)->accept_queue); INIT_LIST_HEAD(&sdp_sk(child)->backlog_queue); - INIT_DELAYED_WORK(&sdp_sk(child)->time_wait_work, sdp_time_wait_work); + INIT_DELAYED_WORK(&sdp_sk(child)->fin_wait_work, sdp_fin_wait_work); INIT_WORK(&sdp_sk(child)->destroy_work, sdp_destroy_work); dst_addr = (struct sockaddr_in *)&id->route.addr.dst_addr; @@ -256,7 +256,7 @@ int sdp_connect_handler(struct sock *sk, struct rdma_cm_id *id, list_add_tail(&sdp_sk(child)->backlog_queue, &sdp_sk(sk)->backlog_queue); sdp_sk(child)->parent = sk; - sdp_set_state(child, TCP_SYN_RECV); + sdp_exch_state(child, TCPF_LISTEN | TCPF_CLOSE, TCP_SYN_RECV); /* child->sk_write_space(child); */ /* child->sk_data_ready(child, 0); */ @@ -272,7 +272,7 @@ static int sdp_response_handler(struct sock *sk, struct rdma_cm_id *id, struct sockaddr_in *dst_addr; sdp_dbg(sk, "%s\n", __func__); - sdp_set_state(sk, TCP_ESTABLISHED); + sdp_exch_state(sk, TCPF_SYN_SENT, TCP_ESTABLISHED); if (sock_flag(sk, SOCK_KEEPOPEN)) sdp_start_keepalive_timer(sk); @@ -316,7 +316,7 @@ int sdp_connected_handler(struct sock *sk, struct rdma_cm_event *event) parent = sdp_sk(sk)->parent; BUG_ON(!parent); - sdp_set_state(sk, TCP_ESTABLISHED); + sdp_exch_state(sk, TCPF_SYN_RECV, TCP_ESTABLISHED); if (sock_flag(sk, SOCK_KEEPOPEN)) sdp_start_keepalive_timer(sk); @@ -500,6 +500,14 @@ int sdp_cma_handler(struct rdma_cm_id *id, struct rdma_cm_event *event) break; case RDMA_CM_EVENT_DISCONNECTED: /* This means DREQ/DREP received */ sdp_dbg(sk, "RDMA_CM_EVENT_DISCONNECTED\n"); + + if (sk->sk_state == TCP_LAST_ACK) { + sdp_exch_state(sk, TCPF_LAST_ACK, TCP_TIME_WAIT); + + sdp_dbg(sk, "%s: waiting for Infiniband tear down\n", + __func__); + } + rdma_disconnect(id); if (sk->sk_state != TCP_TIME_WAIT) { @@ -525,6 +533,12 @@ int sdp_cma_handler(struct rdma_cm_id *id, struct rdma_cm_event *event) sdp_dbg(sk, "%s event %d handled\n", __func__, event->event); if (rc && sdp_sk(sk)->id == id) { + if (sk->sk_state == TCP_SYN_RECV) { + /* sdp_close() will not be called therefore we need + to take a refernce till infiniband teardown is + finished */ + sock_hold(sk); + } child = sk; sdp_sk(sk)->id = NULL; id->qp = NULL; diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c index 960ec90..4cfc4af 100644 --- a/drivers/infiniband/ulp/sdp/sdp_main.c +++ b/drivers/infiniband/ulp/sdp/sdp_main.c @@ -342,10 +342,10 @@ void sdp_reset_sk(struct sock *sk, int rc) memset((void *)&ssk->id, 0, sizeof(*ssk) - offsetof(typeof(*ssk), id)); - queue_work(sdp_workqueue, &ssk->destroy_work); - sk->sk_state_change(sk); + queue_work(sdp_workqueue, &ssk->destroy_work); + read_unlock(&device_removal_lock); } @@ -414,6 +414,12 @@ static void sdp_destruct(struct sock *sk) struct sdp_sock *s, *t; sdp_dbg(sk, "%s\n", __func__); + if (ssk->destructed_already) { + sdp_warn(sk, "redestructing sk!"); + return; + } + + ssk->destructed_already = 1; sdp_remove_sock(ssk); @@ -441,7 +447,7 @@ static void sdp_send_disconnect(struct sock *sk) /* * State processing on a close. - * TCP_ESTABLISHED -> TCP_FIN_WAIT1 -> TCP_FIN_WAIT2 -> TCP_CLOSE + * TCP_ESTABLISHED -> TCP_FIN_WAIT1 -> TCP_CLOSE */ static int sdp_close_state(struct sock *sk) @@ -450,9 +456,9 @@ static int sdp_close_state(struct sock *sk) return 0; if (sk->sk_state == TCP_ESTABLISHED) - sdp_set_state(sk, TCP_FIN_WAIT2);/* should be TCP_FIN_WAIT1 */ + sdp_exch_state(sk, TCPF_ESTABLISHED, TCP_FIN_WAIT1); else if (sk->sk_state == TCP_CLOSE_WAIT) - sdp_set_state(sk, TCP_LAST_ACK); + sdp_exch_state(sk, TCPF_CLOSE_WAIT, TCP_LAST_ACK); else return 0; return 1; @@ -464,27 +470,30 @@ static void sdp_close(struct sock *sk, long timeout) struct sk_buff *skb; int data_was_unread = 0; - if ((1 << sk->sk_state) & (TCPF_TIME_WAIT | TCPF_CLOSE)) { - sock_put(sk); - return; - } - lock_sock(sk); - sock_hold(sk); sdp_dbg(sk, "%s\n", __func__); sdp_delete_keepalive_timer(sk); sk->sk_shutdown = SHUTDOWN_MASK; + + if ((1 << sk->sk_state) & (TCPF_TIME_WAIT | TCPF_CLOSE)) { + /* this could happen if socket was closed by a CM teardown + and after that the user called close() */ + goto out; + } + if (sk->sk_state == TCP_LISTEN || sk->sk_state == TCP_SYN_SENT) { - sdp_set_state(sk, TCP_CLOSE); + sdp_exch_state(sk, TCPF_LISTEN | TCPF_SYN_SENT, TCP_CLOSE); /* Special case: stop listening. This is done by sdp_destruct. */ goto adjudge_to_death; } + sock_hold(sk); + /* We need to flush the recv. buffs. We do this only on the * descriptor close, not protocol-sourced closes, because the * reader process may not have drained the data yet! @@ -505,17 +514,12 @@ static void sdp_close(struct sock *sk, long timeout) * the FTP client, wheee... Note: timeout is always zero * in such a case. */ - if (data_was_unread) { + if (data_was_unread || + (sock_flag(sk, SOCK_LINGER) && !sk->sk_lingertime)) { /* Unread data was tossed, zap the connection. */ NET_INC_STATS_USER(LINUX_MIB_TCPABORTONCLOSE); - sdp_set_state(sk, TCP_TIME_WAIT); - - /* Go into abortive close */ - sk->sk_prot->disconnect(sk, 0); - } else if (sock_flag(sk, SOCK_LINGER) && !sk->sk_lingertime) { - /* Check zero linger _after_ checking for unread data. */ - NET_INC_STATS_USER(LINUX_MIB_TCPABORTONDATA); - sdp_set_state(sk, TCP_TIME_WAIT); + sdp_exch_state(sk, TCPF_CLOSE_WAIT | TCPF_ESTABLISHED, + TCP_TIME_WAIT); /* Go into abortive close */ sk->sk_prot->disconnect(sk, 0); @@ -530,7 +534,8 @@ static void sdp_close(struct sock *sk, long timeout) /* TODO: state should move to CLOSE or CLOSE_WAIT etc on disconnect. Since it currently doesn't, do it here to avoid blocking below. */ if (!sdp_sk(sk)->id) - sdp_set_state(sk, TCP_CLOSE); + sdp_exch_state(sk, TCPF_FIN_WAIT1 | TCPF_LAST_ACK | + TCPF_CLOSE_WAIT, TCP_CLOSE); sk_stream_wait_close(sk, timeout); @@ -557,7 +562,7 @@ adjudge_to_death: * consume significant resources. Let's do it with special * linger2 option. --ANK */ - if ((1 << sk->sk_state) & (TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2)) { + if (sk->sk_state == TCP_FIN_WAIT1) { sdp_sk(sk)->fin_wait_timeout = 1; /* TODO: liger2 unimplemented. We should wait 3.5 * rto. How do I know rto? */ @@ -565,22 +570,16 @@ adjudge_to_death: sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); atomic_inc(sk->sk_prot->orphan_count); - queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, + queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->fin_wait_work, TCP_FIN_TIMEOUT); } /* TODO: limit number of orphaned sockets. TCP has sysctl_tcp_mem and sysctl_tcp_max_orphans */ - if (sk->sk_state != TCP_CLOSE) { - sdp_dbg(sk, "Hold socket till end of Infiniband tear-down\n"); - /* Hold socket till end of Infiniband tear-down */ - sock_hold(sk); - } - +out: release_sock(sk); - sock_put(sk); sk_common_release(sk); } @@ -625,7 +624,7 @@ static int sdp_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) return rc; } - sdp_set_state(sk, TCP_SYN_SENT); + sdp_exch_state(sk, TCPF_CLOSE, TCP_SYN_SENT); return 0; } @@ -646,7 +645,7 @@ static int sdp_disconnect(struct sock *sk, int flags) return rc; } - sdp_set_state(sk, TCP_CLOSE); + sdp_exch_state(sk, TCPF_LISTEN, TCP_CLOSE); id = ssk->id; ssk->id = NULL; release_sock(sk); /* release socket since locking semantics is parent @@ -835,7 +834,7 @@ static int sdp_ioctl(struct sock *sk, int cmd, unsigned long arg) void sdp_cancel_fin_wait_timeout(struct sdp_sock *ssk) { ssk->fin_wait_timeout = 0; - cancel_delayed_work(&ssk->time_wait_work); + cancel_delayed_work(&ssk->fin_wait_work); atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } @@ -848,14 +847,19 @@ void sdp_destroy_work(struct work_struct *work) if (ssk->fin_wait_timeout) sdp_cancel_fin_wait_timeout(ssk); - sdp_set_state(sk, TCP_CLOSE); + if (sk->sk_state == TCP_TIME_WAIT) + sock_put(sk); - sock_put(sk); + /* In normal close current state is TCP_TIME_WAIT or TCP_CLOSE + but if a CM connection is dropped below our legs state could + be any state */ + sdp_exch_state(sk, ~0, TCP_CLOSE); } -void sdp_time_wait_work(struct work_struct *work) +void sdp_fin_wait_work(struct work_struct *work) { - struct sdp_sock *ssk = container_of(work, struct sdp_sock, time_wait_work.work); + struct sdp_sock *ssk = + container_of(work, struct sdp_sock, fin_wait_work.work); struct sock *sk = &ssk->isk.sk; lock_sock(sk); sdp_dbg(sk, "%s\n", __func__); @@ -868,6 +872,7 @@ void sdp_time_wait_work(struct work_struct *work) sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); sdp_sk(sk)->fin_wait_timeout = 0; + sdp_exch_state(sk, TCPF_FIN_WAIT1, TCP_TIME_WAIT); release_sock(sk); atomic_dec(sk->sk_prot->orphan_count); @@ -884,12 +889,14 @@ static int sdp_init_sock(struct sock *sk) INIT_LIST_HEAD(&ssk->accept_queue); INIT_LIST_HEAD(&ssk->backlog_queue); - INIT_DELAYED_WORK(&ssk->time_wait_work, sdp_time_wait_work); + INIT_DELAYED_WORK(&ssk->fin_wait_work, sdp_fin_wait_work); INIT_WORK(&ssk->destroy_work, sdp_destroy_work); sk->sk_route_caps |= NETIF_F_SG | NETIF_F_NO_CSUM; ssk->sdp_disconnect = 0; + ssk->destructed_already = 0; + spin_lock_init(&ssk->lock); return 0; } @@ -902,15 +909,8 @@ static void sdp_shutdown(struct sock *sk, int how) if (!(how & SEND_SHUTDOWN)) return; - if ((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) - return; - - if (sk->sk_state == TCP_ESTABLISHED) - sdp_set_state(sk, TCP_FIN_WAIT2);/* should be TCP_FIN_WAIT1 */ - else if (sk->sk_state == TCP_CLOSE_WAIT) - sdp_set_state(sk, TCP_LAST_ACK); - else - return; + if (!sdp_close_state(sk)) + return; /* * Just turn off CORK here. @@ -2002,7 +2002,7 @@ static int sdp_listen(struct sock *sk, int backlog) sdp_warn(sk, "rdma_listen failed: %d\n", rc); sdp_set_error(sk, rc); } else - sdp_set_state(sk, TCP_LISTEN); + sdp_exch_state(sk, TCPF_CLOSE, TCP_LISTEN); return rc; } diff --git a/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch b/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch index a9d65a7..dccc9b6 100644 --- a/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch +++ b/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch @@ -2,31 +2,31 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== --- ofed_kernel-2.6.9_U4.orig/drivers/infiniband/ulp/sdp/sdp_main.c +++ ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -564,7 +564,6 @@ adjudge_to_death: +@@ -566,7 +566,6 @@ adjudge_to_death: /* TODO: tcp_fin_time to get timeout */ sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - atomic_inc(sk->sk_prot->orphan_count); - queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, + queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->fin_wait_work, TCP_FIN_TIMEOUT); } -@@ -836,7 +835,6 @@ void sdp_cancel_fin_wait_timeout(struct +@@ -832,7 +831,6 @@ void sdp_cancel_fin_wait_timeout(struct { ssk->fin_wait_timeout = 0; - cancel_delayed_work(&ssk->time_wait_work); + cancel_delayed_work(&ssk->fin_wait_work); - atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } void sdp_destroy_work(struct work_struct *work) -@@ -870,7 +868,6 @@ void sdp_time_wait_work(struct work_stru - sdp_sk(sk)->fin_wait_timeout = 0; +@@ -874,7 +872,6 @@ void sdp_fin_wait_work(struct work_struc + sdp_exch_state(sk, TCPF_FIN_WAIT1, TCP_TIME_WAIT); release_sock(sk); - atomic_dec(sk->sk_prot->orphan_count); if (sdp_sk(sk)->id) rdma_disconnect(sdp_sk(sk)->id); -@@ -2101,7 +2098,6 @@ void sdp_urg(struct sdp_sock *ssk, struc +@@ -2099,7 +2096,6 @@ void sdp_urg(struct sdp_sock *ssk, struc static atomic_t sockets_allocated; static atomic_t memory_allocated; @@ -34,7 +34,7 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c static int memory_pressure; struct proto sdp_proto = { .close = sdp_close, -@@ -2122,13 +2118,11 @@ struct proto sdp_proto = { +@@ -2120,13 +2116,11 @@ struct proto sdp_proto = { .sockets_allocated = &sockets_allocated, .memory_allocated = &memory_allocated, .memory_pressure = &memory_pressure, @@ -50,7 +50,7 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c .name = "SDP", }; -@@ -2490,9 +2484,6 @@ static void __exit sdp_exit(void) +@@ -2488,9 +2482,6 @@ static void __exit sdp_exit(void) sock_unregister(PF_INET_SDP); proto_unregister(&sdp_proto); diff --git a/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch b/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch index a9d65a7..dccc9b6 100644 --- a/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch +++ b/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch @@ -2,31 +2,31 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== --- ofed_kernel-2.6.9_U4.orig/drivers/infiniband/ulp/sdp/sdp_main.c +++ ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -564,7 +564,6 @@ adjudge_to_death: +@@ -566,7 +566,6 @@ adjudge_to_death: /* TODO: tcp_fin_time to get timeout */ sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - atomic_inc(sk->sk_prot->orphan_count); - queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, + queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->fin_wait_work, TCP_FIN_TIMEOUT); } -@@ -836,7 +835,6 @@ void sdp_cancel_fin_wait_timeout(struct +@@ -832,7 +831,6 @@ void sdp_cancel_fin_wait_timeout(struct { ssk->fin_wait_timeout = 0; - cancel_delayed_work(&ssk->time_wait_work); + cancel_delayed_work(&ssk->fin_wait_work); - atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } void sdp_destroy_work(struct work_struct *work) -@@ -870,7 +868,6 @@ void sdp_time_wait_work(struct work_stru - sdp_sk(sk)->fin_wait_timeout = 0; +@@ -874,7 +872,6 @@ void sdp_fin_wait_work(struct work_struc + sdp_exch_state(sk, TCPF_FIN_WAIT1, TCP_TIME_WAIT); release_sock(sk); - atomic_dec(sk->sk_prot->orphan_count); if (sdp_sk(sk)->id) rdma_disconnect(sdp_sk(sk)->id); -@@ -2101,7 +2098,6 @@ void sdp_urg(struct sdp_sock *ssk, struc +@@ -2099,7 +2096,6 @@ void sdp_urg(struct sdp_sock *ssk, struc static atomic_t sockets_allocated; static atomic_t memory_allocated; @@ -34,7 +34,7 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c static int memory_pressure; struct proto sdp_proto = { .close = sdp_close, -@@ -2122,13 +2118,11 @@ struct proto sdp_proto = { +@@ -2120,13 +2116,11 @@ struct proto sdp_proto = { .sockets_allocated = &sockets_allocated, .memory_allocated = &memory_allocated, .memory_pressure = &memory_pressure, @@ -50,7 +50,7 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c .name = "SDP", }; -@@ -2490,9 +2484,6 @@ static void __exit sdp_exit(void) +@@ -2488,9 +2482,6 @@ static void __exit sdp_exit(void) sock_unregister(PF_INET_SDP); proto_unregister(&sdp_proto); diff --git a/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch index 8e4b929..22141d2 100644 --- a/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch @@ -1,7 +1,8 @@ -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 -@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_bcopy.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_bcopy.c +@@ -134,7 +134,7 @@ static void sdp_fin(struct sock *sk) } @@ -10,7 +11,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -185,7 +185,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -19,7 +20,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, +@@ -229,7 +229,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -28,7 +29,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc +@@ -299,11 +299,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -42,7 +43,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss +@@ -471,7 +471,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -51,7 +52,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -509,7 +509,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -60,7 +61,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -534,7 +534,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -69,16 +70,16 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -556,7 +556,7 @@ void sdp_post_sends(struct sdp_sock *ssk likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && likely((1 << ssk->isk.sk.sk_state) & - (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk +@@ -568,7 +568,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { ssk->sdp_disconnect = 0; @@ -87,7 +88,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s +@@ -765,7 +765,7 @@ static int sdp_handle_send_comp(struct s } out: @@ -96,7 +97,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp return 0; } -@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) +@@ -851,7 +851,7 @@ void sdp_work(struct work_struct *work) sdp_poll_cq(ssk, cq); release_sock(sk); @@ -105,9 +106,10 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_cma.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -117,10 +119,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/s qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h ---- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 -@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp.h ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp.h +@@ -319,30 +319,4 @@ void sdp_post_keepalive(struct sdp_sock void sdp_start_keepalive_timer(struct sock *sk); void sdp_bzcopy_write_space(struct sdp_sock *ssk); @@ -151,10 +154,11 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h - - #endif -diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c ---- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 -@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l +Index: linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +=================================================================== +--- linux_2.6.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ linux_2.6/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -501,7 +501,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -163,7 +167,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s +@@ -1195,7 +1195,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -172,7 +176,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } } -@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc +@@ -1212,8 +1212,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); @@ -182,7 +186,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s +@@ -1377,7 +1376,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -191,7 +195,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; if (!page) { -@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct +@@ -1449,7 +1448,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -200,7 +204,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1659,8 +1658,8 @@ new_segment: +@@ -1657,8 +1656,8 @@ new_segment: goto wait_for_sndbuf; } @@ -211,7 +215,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ if (!skb) goto wait_for_memory; -@@ -1684,7 +1683,7 @@ new_segment: +@@ -1682,7 +1681,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -220,7 +224,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1760,7 +1759,7 @@ do_fault: +@@ -1758,7 +1757,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -229,7 +233,7 @@ diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/ } do_error: -@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) +@@ -2363,6 +2362,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch b/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch index a9d65a7..dccc9b6 100644 --- a/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch +++ b/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch @@ -2,31 +2,31 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== --- ofed_kernel-2.6.9_U4.orig/drivers/infiniband/ulp/sdp/sdp_main.c +++ ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -564,7 +564,6 @@ adjudge_to_death: +@@ -566,7 +566,6 @@ adjudge_to_death: /* TODO: tcp_fin_time to get timeout */ sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - atomic_inc(sk->sk_prot->orphan_count); - queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, + queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->fin_wait_work, TCP_FIN_TIMEOUT); } -@@ -836,7 +835,6 @@ void sdp_cancel_fin_wait_timeout(struct +@@ -832,7 +831,6 @@ void sdp_cancel_fin_wait_timeout(struct { ssk->fin_wait_timeout = 0; - cancel_delayed_work(&ssk->time_wait_work); + cancel_delayed_work(&ssk->fin_wait_work); - atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } void sdp_destroy_work(struct work_struct *work) -@@ -870,7 +868,6 @@ void sdp_time_wait_work(struct work_stru - sdp_sk(sk)->fin_wait_timeout = 0; +@@ -874,7 +872,6 @@ void sdp_fin_wait_work(struct work_struc + sdp_exch_state(sk, TCPF_FIN_WAIT1, TCP_TIME_WAIT); release_sock(sk); - atomic_dec(sk->sk_prot->orphan_count); if (sdp_sk(sk)->id) rdma_disconnect(sdp_sk(sk)->id); -@@ -2101,7 +2098,6 @@ void sdp_urg(struct sdp_sock *ssk, struc +@@ -2099,7 +2096,6 @@ void sdp_urg(struct sdp_sock *ssk, struc static atomic_t sockets_allocated; static atomic_t memory_allocated; @@ -34,7 +34,7 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c static int memory_pressure; struct proto sdp_proto = { .close = sdp_close, -@@ -2122,13 +2118,11 @@ struct proto sdp_proto = { +@@ -2120,13 +2116,11 @@ struct proto sdp_proto = { .sockets_allocated = &sockets_allocated, .memory_allocated = &memory_allocated, .memory_pressure = &memory_pressure, @@ -50,7 +50,7 @@ Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c .name = "SDP", }; -@@ -2490,9 +2484,6 @@ static void __exit sdp_exit(void) +@@ -2488,9 +2482,6 @@ static void __exit sdp_exit(void) sock_unregister(PF_INET_SDP); proto_unregister(&sdp_proto); -- 1.5.3 From amirv at mellanox.co.il Thu Jul 10 00:48:52 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 10 Jul 2008 10:48:52 +0300 Subject: [ofa-general] [PATCH v2 5/6] Changed after fix to sdp disconnect In-Reply-To: <12156761331845-git-send-email-amirv@mellanox.co.il> References: <12156761331725-git-send-email-amirv@mellanox.co.il> <12156761332878-git-send-email-amirv@mellanox.co.il> <12156761333757-git-send-email-amirv@mellanox.co.il> <12156761331845-git-send-email-amirv@mellanox.co.il> Message-ID: <12156761333073-git-send-email-amirv@mellanox.co.il> Signed-off-by: Amir Vadai --- .../2.6.16/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.17/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.18/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.19/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.20/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.21/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.22/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.23/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.24/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../2.6.9_U4/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../backport/2.6.9_U4/sdp_7277_to_2_6_11.patch | 40 +++--- .../2.6.9_U5/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../backport/2.6.9_U5/sdp_7277_to_2_6_11.patch | 40 +++--- .../2.6.9_U6/sdp_0090_revert_to_2_6_24.patch | 162 ++++++++++---------- .../backport/2.6.9_U6/sdp_7277_to_2_6_11.patch | 40 +++--- 23 files changed, 1640 insertions(+), 1720 deletions(-) diff --git a/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16_sles10/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16_sles10_sp1/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.16_sles10_sp2/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.17/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18-EL5.1/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18-EL5.2/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18_FC6/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.18_suse10_2/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.19/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.20/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.21/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.22/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.22_suse10_3/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.23/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.24/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.9_U4/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch b/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch index 789362b..a9d65a7 100644 --- a/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch +++ b/kernel_patches/backport/2.6.9_U4/sdp_7277_to_2_6_11.patch @@ -1,32 +1,32 @@ -Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c +Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== ---- openib_gen2-20060517-1739_check.orig/drivers/infiniband/ulp/sdp/sdp_main.c 2006-05-17 17:45:32.000000000 +0300 -+++ openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c 2006-05-17 18:33:13.000000000 +0300 -@@ -300,7 +300,6 @@ +--- ofed_kernel-2.6.9_U4.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -564,7 +564,6 @@ adjudge_to_death: /* TODO: tcp_fin_time to get timeout */ sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - atomic_inc(sk->sk_prot->orphan_count); queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, TCP_FIN_TIMEOUT); - goto out; -@@ -495,7 +494,6 @@ - sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - - cancel_delayed_work(&sdp_sk(sk)->time_wait_work); -- atomic_dec(sk->sk_prot->orphan_count); - - sock_put(sk); + } +@@ -836,7 +835,6 @@ void sdp_cancel_fin_wait_timeout(struct + { + ssk->fin_wait_timeout = 0; + cancel_delayed_work(&ssk->time_wait_work); +- atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } -@@ -517,7 +515,6 @@ - sdp_sk(sk)->time_wait = 0; + + void sdp_destroy_work(struct work_struct *work) +@@ -870,7 +868,6 @@ void sdp_time_wait_work(struct work_stru + sdp_sk(sk)->fin_wait_timeout = 0; release_sock(sk); - atomic_dec(sk->sk_prot->orphan_count); - sock_put(sk); - } -@@ -1166,7 +1163,6 @@ + if (sdp_sk(sk)->id) + rdma_disconnect(sdp_sk(sk)->id); +@@ -2101,7 +2098,6 @@ void sdp_urg(struct sdp_sock *ssk, struc static atomic_t sockets_allocated; static atomic_t memory_allocated; @@ -34,7 +34,7 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c static int memory_pressure; struct proto sdp_proto = { .close = sdp_close, -@@ -1187,13 +1183,11 @@ +@@ -2122,13 +2118,11 @@ struct proto sdp_proto = { .sockets_allocated = &sockets_allocated, .memory_allocated = &memory_allocated, .memory_pressure = &memory_pressure, @@ -50,7 +50,7 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c .name = "SDP", }; -@@ -1298,9 +1292,6 @@ +@@ -2490,9 +2484,6 @@ static void __exit sdp_exit(void) sock_unregister(PF_INET_SDP); proto_unregister(&sdp_proto); @@ -59,4 +59,4 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c - atomic_read(&orphan_count)); destroy_workqueue(sdp_workqueue); flush_scheduled_work(); - } + diff --git a/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.9_U5/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch b/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch index 789362b..a9d65a7 100644 --- a/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch +++ b/kernel_patches/backport/2.6.9_U5/sdp_7277_to_2_6_11.patch @@ -1,32 +1,32 @@ -Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c +Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== ---- openib_gen2-20060517-1739_check.orig/drivers/infiniband/ulp/sdp/sdp_main.c 2006-05-17 17:45:32.000000000 +0300 -+++ openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c 2006-05-17 18:33:13.000000000 +0300 -@@ -300,7 +300,6 @@ +--- ofed_kernel-2.6.9_U4.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -564,7 +564,6 @@ adjudge_to_death: /* TODO: tcp_fin_time to get timeout */ sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - atomic_inc(sk->sk_prot->orphan_count); queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, TCP_FIN_TIMEOUT); - goto out; -@@ -495,7 +494,6 @@ - sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - - cancel_delayed_work(&sdp_sk(sk)->time_wait_work); -- atomic_dec(sk->sk_prot->orphan_count); - - sock_put(sk); + } +@@ -836,7 +835,6 @@ void sdp_cancel_fin_wait_timeout(struct + { + ssk->fin_wait_timeout = 0; + cancel_delayed_work(&ssk->time_wait_work); +- atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } -@@ -517,7 +515,6 @@ - sdp_sk(sk)->time_wait = 0; + + void sdp_destroy_work(struct work_struct *work) +@@ -870,7 +868,6 @@ void sdp_time_wait_work(struct work_stru + sdp_sk(sk)->fin_wait_timeout = 0; release_sock(sk); - atomic_dec(sk->sk_prot->orphan_count); - sock_put(sk); - } -@@ -1166,7 +1163,6 @@ + if (sdp_sk(sk)->id) + rdma_disconnect(sdp_sk(sk)->id); +@@ -2101,7 +2098,6 @@ void sdp_urg(struct sdp_sock *ssk, struc static atomic_t sockets_allocated; static atomic_t memory_allocated; @@ -34,7 +34,7 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c static int memory_pressure; struct proto sdp_proto = { .close = sdp_close, -@@ -1187,13 +1183,11 @@ +@@ -2122,13 +2118,11 @@ struct proto sdp_proto = { .sockets_allocated = &sockets_allocated, .memory_allocated = &memory_allocated, .memory_pressure = &memory_pressure, @@ -50,7 +50,7 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c .name = "SDP", }; -@@ -1298,9 +1292,6 @@ +@@ -2490,9 +2484,6 @@ static void __exit sdp_exit(void) sock_unregister(PF_INET_SDP); proto_unregister(&sdp_proto); @@ -59,4 +59,4 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c - atomic_read(&orphan_count)); destroy_workqueue(sdp_workqueue); flush_scheduled_work(); - } + diff --git a/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch b/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch index 78550f8..8e4b929 100644 --- a/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch +++ b/kernel_patches/backport/2.6.9_U6/sdp_0090_revert_to_2_6_24.patch @@ -1,44 +1,8 @@ -diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h -index c434b60..e4d3eff 100644 ---- a/drivers/infiniband/ulp/sdp/sdp.h -+++ b/drivers/infiniband/ulp/sdp/sdp.h -@@ -265,30 +265,4 @@ void sdp_post_keepalive(struct sdp_sock *ssk); - void sdp_start_keepalive_timer(struct sock *sk); - void sdp_bzcopy_write_space(struct sdp_sock *ssk); - --static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) --{ -- struct sk_buff *skb; -- -- /* The TCP header must be at least 32-bit aligned. */ -- size = ALIGN(size, 4); -- -- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); -- if (skb) { -- if (sk_wmem_schedule(sk, skb->truesize)) { -- /* -- * Make sure that we have exactly size bytes -- * available to the caller, no more, no less. -- */ -- skb_reserve(skb, skb_tailroom(skb) - size); -- return skb; -- } -- __kfree_skb(skb); -- } else { -- sk->sk_prot->enter_memory_pressure(); -- sk_stream_moderate_sndbuf(sk); -- } -- return NULL; --} -- -- - #endif -diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -index 36cbbad..ad788f7 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c -+++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c -@@ -105,7 +105,7 @@ static void sdp_fin(struct sock *sk) - sock_set_flag(sk, SOCK_DONE); +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c +--- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c 2008-06-22 15:02:31.000000000 +0300 +@@ -141,7 +141,7 @@ static void sdp_fin(struct sock *sk) + } - sk_mem_reclaim(sk); @@ -46,7 +10,7 @@ index 36cbbad..ad788f7 100644 if (!sock_flag(sk, SOCK_DEAD)) { sk->sk_state_change(sk); -@@ -156,7 +156,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -192,7 +192,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct ib_send_wr *bad_wr; h->mid = mid; @@ -55,7 +19,7 @@ index 36cbbad..ad788f7 100644 h->flags = SDP_OOB_PRES | SDP_OOB_PEND; else h->flags = 0; -@@ -200,7 +200,7 @@ void sdp_post_send(struct sdp_sock *ssk, struct sk_buff *skb, u8 mid) +@@ -236,7 +236,7 @@ void sdp_post_send(struct sdp_sock *ssk, ssk->tx_wr.num_sge = frags + 1; ssk->tx_wr.opcode = IB_WR_SEND; ssk->tx_wr.send_flags = IB_SEND_SIGNALED; @@ -64,7 +28,7 @@ index 36cbbad..ad788f7 100644 ssk->tx_wr.send_flags |= IB_SEND_SOLICITED; rc = ib_post_send(ssk->qp, &ssk->tx_wr, &bad_wr); ++ssk->tx_head; -@@ -270,11 +270,11 @@ static void sdp_post_recv(struct sdp_sock *ssk) +@@ -306,11 +306,11 @@ static void sdp_post_recv(struct sdp_soc /* TODO: allocate from cache */ if (unlikely(ssk->isk.sk.sk_allocation)) { @@ -78,7 +42,7 @@ index 36cbbad..ad788f7 100644 GFP_KERNEL); gfp_page = GFP_HIGHUSER; } -@@ -442,7 +442,7 @@ int sdp_post_credits(struct sdp_sock *ssk) +@@ -478,7 +478,7 @@ int sdp_post_credits(struct sdp_sock *ss if (likely(ssk->bufs > 1) && likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { struct sk_buff *skb; @@ -87,7 +51,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh), GFP_KERNEL); if (!skb) -@@ -480,7 +480,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -516,7 +516,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *resp_size; ssk->recv_request = 0; @@ -96,7 +60,7 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*resp_size), gfp_page); -@@ -505,7 +505,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) +@@ -541,7 +541,7 @@ void sdp_post_sends(struct sdp_sock *ssk ssk->tx_head > ssk->sent_request_head + SDP_RESIZE_WAIT && ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) { struct sdp_chrecvbuf *req_size; @@ -105,35 +69,35 @@ index 36cbbad..ad788f7 100644 sizeof(struct sdp_bsdh) + sizeof(*req_size), gfp_page); -@@ -525,7 +525,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - if (unlikely(c < ssk->rx_head - ssk->rx_tail) && - likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { +@@ -563,7 +563,7 @@ void sdp_post_sends(struct sdp_sock *ssk + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); /* FIXME */ -@@ -537,7 +537,7 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && +@@ -575,7 +575,7 @@ void sdp_post_sends(struct sdp_sock *ssk !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; - skb = sdp_stream_alloc_skb(&ssk->isk.sk, + skb = sk_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ -@@ -684,7 +684,7 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) - skb = sdp_send_completion(ssk, wc->wr_id); - if (unlikely(!skb)) - return; -- sk_wmem_free_skb(&ssk->isk.sk, skb); -+ sk_stream_free_skb(&ssk->isk.sk, skb); - if (unlikely(wc->status)) { - if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, -@@ -766,7 +766,7 @@ void sdp_work(struct work_struct *work) - goto out; +@@ -788,7 +788,7 @@ static int sdp_handle_send_comp(struct s + } + + out: +- sk_wmem_free_skb(&ssk->isk.sk, skb); ++ sk_stream_free_skb(&ssk->isk.sk, skb); + + return 0; + } +@@ -874,7 +874,7 @@ void sdp_work(struct work_struct *work) + sdp_poll_cq(ssk, cq); release_sock(sk); - sk_mem_reclaim(sk); @@ -141,11 +105,10 @@ index 36cbbad..ad788f7 100644 lock_sock(sk); cq = ssk->cq; if (unlikely(!cq)) -diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c -index b9f54d0..0cab38b 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_cma.c -+++ b/drivers/infiniband/ulp/sdp/sdp_cma.c -@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct rdma_cm_id *id) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c +--- a/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_cma.c 2008-06-22 15:02:31.000000000 +0300 +@@ -162,8 +162,6 @@ int sdp_init_qp(struct sock *sk, struct goto err_cq; } @@ -154,11 +117,44 @@ index b9f54d0..0cab38b 100644 qp_init_attr.send_cq = qp_init_attr.recv_cq = cq; rc = rdma_create_qp(id, pd, &qp_init_attr); -diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c -index d35c803..dcc60e3 100644 ---- a/drivers/infiniband/ulp/sdp/sdp_main.c -+++ b/drivers/infiniband/ulp/sdp/sdp_main.c -@@ -490,7 +490,7 @@ static void sdp_close(struct sock *sk, long timeout) +diff -Naup a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h +--- a/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp.h 2008-06-22 15:02:31.000000000 +0300 +@@ -266,30 +266,4 @@ void sdp_post_keepalive(struct sdp_sock + void sdp_start_keepalive_timer(struct sock *sk); + void sdp_bzcopy_write_space(struct sdp_sock *ssk); + +-static inline struct sk_buff *sdp_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp) +-{ +- struct sk_buff *skb; +- +- /* The TCP header must be at least 32-bit aligned. */ +- size = ALIGN(size, 4); +- +- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); +- if (skb) { +- if (sk_wmem_schedule(sk, skb->truesize)) { +- /* +- * Make sure that we have exactly size bytes +- * available to the caller, no more, no less. +- */ +- skb_reserve(skb, skb_tailroom(skb) - size); +- return skb; +- } +- __kfree_skb(skb); +- } else { +- sk->sk_prot->enter_memory_pressure(); +- sk_stream_moderate_sndbuf(sk); +- } +- return NULL; +-} +- +- + #endif +diff -Naup a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c +--- a/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:25.000000000 +0300 ++++ b/drivers/infiniband/ulp/sdp/sdp_main.c 2008-06-22 15:02:31.000000000 +0300 +@@ -494,7 +494,7 @@ static void sdp_close(struct sock *sk, l __kfree_skb(skb); } @@ -167,7 +163,7 @@ index d35c803..dcc60e3 100644 /* As outlined in draft-ietf-tcpimpl-prob-03.txt, section * 3.10, we send a RST here because data was lost. To -@@ -1185,7 +1185,7 @@ static inline void sdp_mark_urg(struct sock *sk, struct sdp_sock *ssk, int flags +@@ -1197,7 +1197,7 @@ static inline void sdp_mark_urg(struct s { if (unlikely(flags & MSG_OOB)) { struct sk_buff *skb = sk->sk_write_queue.prev; @@ -176,17 +172,17 @@ index d35c803..dcc60e3 100644 } } -@@ -1202,8 +1202,7 @@ static inline void skb_entail(struct sock *sk, struct sdp_sock *ssk, +@@ -1214,8 +1214,7 @@ static inline void skb_entail(struct soc { skb_header_release(skb); __skb_queue_tail(&sk->sk_write_queue, skb); - sk->sk_wmem_queued += skb->truesize; - sk_mem_charge(sk, skb->truesize); -+ sk_charge_skb(sk, skb); ++ sk_charge_skb(sk, skb); if (!sk->sk_send_head) sk->sk_send_head = skb; if (ssk->nonagle & TCP_NAGLE_PUSH) -@@ -1367,7 +1366,7 @@ static inline int sdp_bcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1379,7 +1378,7 @@ static inline int sdp_bcopy_get(struct s if (copy > PAGE_SIZE - off) copy = PAGE_SIZE - off; @@ -195,7 +191,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; if (!page) { -@@ -1439,7 +1438,7 @@ static inline int sdp_bzcopy_get(struct sock *sk, struct sk_buff *skb, +@@ -1451,7 +1450,7 @@ static inline int sdp_bzcopy_get(struct if (left <= this_page) this_page = left; @@ -204,7 +200,7 @@ index d35c803..dcc60e3 100644 return SDP_DO_WAIT_MEM; skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags, -@@ -1647,8 +1646,8 @@ new_segment: +@@ -1659,8 +1658,8 @@ new_segment: goto wait_for_sndbuf; } @@ -215,7 +211,7 @@ index d35c803..dcc60e3 100644 if (!skb) goto wait_for_memory; -@@ -1672,7 +1671,7 @@ new_segment: +@@ -1684,7 +1683,7 @@ new_segment: /* OOB data byte should be the last byte of the data payload */ @@ -224,7 +220,7 @@ index d35c803..dcc60e3 100644 !(flags & MSG_OOB)) { sdp_mark_push(ssk, skb); goto new_segment; -@@ -1748,7 +1747,7 @@ do_fault: +@@ -1760,7 +1759,7 @@ do_fault: if (sk->sk_send_head == skb) sk->sk_send_head = NULL; __skb_unlink(skb, &sk->sk_write_queue); @@ -233,7 +229,7 @@ index d35c803..dcc60e3 100644 } do_error: -@@ -2353,6 +2352,10 @@ static int __init sdp_proc_init(void) +@@ -2365,6 +2364,10 @@ static int __init sdp_proc_init(void) sdp_seq_afinfo.seq_fops); if (p) p->data = &sdp_seq_afinfo; diff --git a/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch b/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch index 789362b..a9d65a7 100644 --- a/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch +++ b/kernel_patches/backport/2.6.9_U6/sdp_7277_to_2_6_11.patch @@ -1,32 +1,32 @@ -Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c +Index: ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== ---- openib_gen2-20060517-1739_check.orig/drivers/infiniband/ulp/sdp/sdp_main.c 2006-05-17 17:45:32.000000000 +0300 -+++ openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c 2006-05-17 18:33:13.000000000 +0300 -@@ -300,7 +300,6 @@ +--- ofed_kernel-2.6.9_U4.orig/drivers/infiniband/ulp/sdp/sdp_main.c ++++ ofed_kernel-2.6.9_U4/drivers/infiniband/ulp/sdp/sdp_main.c +@@ -564,7 +564,6 @@ adjudge_to_death: /* TODO: tcp_fin_time to get timeout */ sdp_dbg(sk, "%s: entering time wait refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - atomic_inc(sk->sk_prot->orphan_count); queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, TCP_FIN_TIMEOUT); - goto out; -@@ -495,7 +494,6 @@ - sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - - cancel_delayed_work(&sdp_sk(sk)->time_wait_work); -- atomic_dec(sk->sk_prot->orphan_count); - - sock_put(sk); + } +@@ -836,7 +835,6 @@ void sdp_cancel_fin_wait_timeout(struct + { + ssk->fin_wait_timeout = 0; + cancel_delayed_work(&ssk->time_wait_work); +- atomic_dec(ssk->isk.sk.sk_prot->orphan_count); } -@@ -517,7 +515,6 @@ - sdp_sk(sk)->time_wait = 0; + + void sdp_destroy_work(struct work_struct *work) +@@ -870,7 +868,6 @@ void sdp_time_wait_work(struct work_stru + sdp_sk(sk)->fin_wait_timeout = 0; release_sock(sk); - atomic_dec(sk->sk_prot->orphan_count); - sock_put(sk); - } -@@ -1166,7 +1163,6 @@ + if (sdp_sk(sk)->id) + rdma_disconnect(sdp_sk(sk)->id); +@@ -2101,7 +2098,6 @@ void sdp_urg(struct sdp_sock *ssk, struc static atomic_t sockets_allocated; static atomic_t memory_allocated; @@ -34,7 +34,7 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c static int memory_pressure; struct proto sdp_proto = { .close = sdp_close, -@@ -1187,13 +1183,11 @@ +@@ -2122,13 +2118,11 @@ struct proto sdp_proto = { .sockets_allocated = &sockets_allocated, .memory_allocated = &memory_allocated, .memory_pressure = &memory_pressure, @@ -50,7 +50,7 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c .name = "SDP", }; -@@ -1298,9 +1292,6 @@ +@@ -2490,9 +2484,6 @@ static void __exit sdp_exit(void) sock_unregister(PF_INET_SDP); proto_unregister(&sdp_proto); @@ -59,4 +59,4 @@ Index: openib_gen2-20060517-1739_check/drivers/infiniband/ulp/sdp/sdp_main.c - atomic_read(&orphan_count)); destroy_workqueue(sdp_workqueue); flush_scheduled_work(); - } + -- 1.5.3 From amirv at mellanox.co.il Thu Jul 10 00:48:49 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 10 Jul 2008 10:48:49 +0300 Subject: [ofa-general] [PATCH v2 2/6] Add support for event IB_CM_TIMEWAIT_EXIT In-Reply-To: <12156761331725-git-send-email-amirv@mellanox.co.il> References: <12156761331725-git-send-email-amirv@mellanox.co.il> Message-ID: <12156761332878-git-send-email-amirv@mellanox.co.il> Notify the ULP when a IB_CM_TIMEWAIT_EXIT occurs. Signed-off-by: Amir Vadai --- .../fixes/cma_0050_timewait_finished_event.patch | 45 ++++++++++++++++++++ 1 files changed, 45 insertions(+), 0 deletions(-) create mode 100644 kernel_patches/fixes/cma_0050_timewait_finished_event.patch diff --git a/kernel_patches/fixes/cma_0050_timewait_finished_event.patch b/kernel_patches/fixes/cma_0050_timewait_finished_event.patch new file mode 100644 index 0000000..d987529 --- /dev/null +++ b/kernel_patches/fixes/cma_0050_timewait_finished_event.patch @@ -0,0 +1,45 @@ +SDP needs to be notified when CM exits the TIMEWAIT state. +CM does notify the CMA but CMA doesn't pass it to the upper layer. + +Below is a patch for the CMA code. I wasn't sure if it is ok to set +CMA_DISCONNECT in the IB_CM_TIMEWAIT_EXIT instead of doing it in the +IB_CM_DREQ_RECEIVED/IB_CM_DREP_RECEIVED as it is done today. Instead +I changed the check at the beginning of cma_ib_handler() as you can +see in the patch. + +Signed-off-by: Amir Vadai + +--- a/include/rdma/rdma_cm.h 2008-06-18 15:04:22.560904000 +0300 ++++ b/include/rdma/rdma_cm.h 2008-06-11 11:55:26.758053000 +0300 +@@ -53,7 +53,8 @@ enum rdma_cm_event_type { + RDMA_CM_EVENT_DISCONNECTED, + RDMA_CM_EVENT_DEVICE_REMOVAL, + RDMA_CM_EVENT_MULTICAST_JOIN, +- RDMA_CM_EVENT_MULTICAST_ERROR ++ RDMA_CM_EVENT_MULTICAST_ERROR, ++ RDMA_CM_EVENT_TIMWAIT_EXIT + }; + + enum rdma_port_space { +--- a/drivers/infiniband/core/cma.c 2008-06-11 11:24:38.021543000 +0300 ++++ b/drivers/infiniband/core/cma.c 2008-06-18 13:27:08.098747000 +0300 +@@ -931,7 +931,8 @@ static int cma_ib_handler(struct ib_cm_i + struct rdma_cm_event event; + int ret = 0; + +- if (cma_disable_remove(id_priv, CMA_CONNECT)) ++ if ((ib_event->event != IB_CM_TIMEWAIT_EXIT && cma_disable_remove(id_priv, CMA_CONNECT)) || ++ (ib_event->event == IB_CM_TIMEWAIT_EXIT && cma_disable_remove(id_priv, CMA_DISCONNECT))) + return 0; + + memset(&event, 0, sizeof event); +@@ -967,6 +968,8 @@ static int cma_ib_handler(struct ib_cm_i + event.event = RDMA_CM_EVENT_DISCONNECTED; + break; + case IB_CM_TIMEWAIT_EXIT: ++ event.event = RDMA_CM_EVENT_TIMWAIT_EXIT; ++ break; + case IB_CM_MRA_RECEIVED: + /* ignore event */ + goto out; + -- 1.5.3 From amirv at mellanox.co.il Thu Jul 10 00:48:51 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 10 Jul 2008 10:48:51 +0300 Subject: [ofa-general] [PATCH v2 4/6] do gracefull close instead of always doing abortive close. In-Reply-To: <12156761333757-git-send-email-amirv@mellanox.co.il> References: <12156761331725-git-send-email-amirv@mellanox.co.il> <12156761332878-git-send-email-amirv@mellanox.co.il> <12156761333757-git-send-email-amirv@mellanox.co.il> Message-ID: <12156761331845-git-send-email-amirv@mellanox.co.il> Main changes: 1. when a close/shutdown syscall is called, instead of sending a DREQ, put last socket ref count and go to TCP_CLOSE state do: - take a socket reference count - set state to TCP_TIME_WAIT - start infiniband tear down - wait till got RDMA_CM_EVENT_TIMEWAIT_EXIT - set socket state to TCP_CLOSE - put last socket ref count - this will call sdp_destruct() 2. Use fin_wait_timeout to timeout a half closed connection which the peer didn't respond with SDP_MID_DISCONNECT 3. No need for sdp_time_wait 4. Abortive close will immedietly start infiniband teardown - will finilize the socket closing when CM finish. 5. when sdp_post_sends send SDP_MID_DISCONNECT according to socket state multiple DISCONNECT could be sent - changed it to be triggered by a flag. Signed-off-by: Amir Vadai --- drivers/infiniband/ulp/sdp/sdp.h | 20 +++--- drivers/infiniband/ulp/sdp/sdp_bcopy.c | 137 +++++++++++++++++++++++++------- drivers/infiniband/ulp/sdp/sdp_cma.c | 10 ++- drivers/infiniband/ulp/sdp/sdp_main.c | 97 +++++++++++++---------- 4 files changed, 180 insertions(+), 84 deletions(-) diff --git a/drivers/infiniband/ulp/sdp/sdp.h b/drivers/infiniband/ulp/sdp/sdp.h index 5bd4041..3b2dffb 100644 --- a/drivers/infiniband/ulp/sdp/sdp.h +++ b/drivers/infiniband/ulp/sdp/sdp.h @@ -125,7 +125,7 @@ struct sdp_sock { int xmit_size_goal; int nonagle; - int time_wait; + int fin_wait_timeout; unsigned keepalive_time; @@ -133,6 +133,8 @@ struct sdp_sock { unsigned keepalive_tx_head; unsigned keepalive_rx_head; + int sdp_disconnect; + /* Data below will be reset on error */ /* rdma specific */ struct rdma_cm_id *id; @@ -227,15 +229,12 @@ static inline void sdp_set_error(struct sock *sk, int err) { sk->sk_err = -err; if (sk->sk_socket) - sk->sk_socket->state = SS_UNCONNECTED; - - sdp_set_state(sk, TCP_CLOSE); + sk->sk_socket->state = SS_DISCONNECTING; - if (sdp_sk(sk)->time_wait) { - sdp_dbg(sk, "%s: destroy in time wait state\n", __func__); - sdp_sk(sk)->time_wait = 0; - queue_work(sdp_workqueue, &sdp_sk(sk)->destroy_work); - } + if (sk->sk_state == TCP_SYN_SENT) + sdp_set_state(sk, TCP_CLOSE); + else + sdp_set_state(sk, TCP_TIME_WAIT); sk->sk_error_report(sk); } @@ -245,7 +244,6 @@ extern struct workqueue_struct *sdp_workqueue; int sdp_cma_handler(struct rdma_cm_id *, struct rdma_cm_event *); void sdp_reset(struct sock *sk); void sdp_reset_sk(struct sock *sk, int rc); -void sdp_time_wait_destroy_sk(struct sdp_sock *ssk); void sdp_completion_handler(struct ib_cq *cq, void *cq_context); void sdp_work(struct work_struct *work); int sdp_post_credits(struct sdp_sock *ssk); @@ -254,6 +252,8 @@ void sdp_post_recvs(struct sdp_sock *ssk); int sdp_poll_cq(struct sdp_sock *ssk, struct ib_cq *cq); void sdp_post_sends(struct sdp_sock *ssk, int nonagle); void sdp_destroy_work(struct work_struct *work); +void sdp_cancel_fin_wait_timeout(struct sdp_sock *ssk); +void sdp_fin_work(struct work_struct *work); void sdp_time_wait_work(struct work_struct *work); struct sk_buff *sdp_recv_completion(struct sdp_sock *ssk, int id); struct sk_buff *sdp_send_completion(struct sdp_sock *ssk, int mseq); diff --git a/drivers/infiniband/ulp/sdp/sdp_bcopy.c b/drivers/infiniband/ulp/sdp/sdp_bcopy.c index 3a8d5ac..d3f50b5 100644 --- a/drivers/infiniband/ulp/sdp/sdp_bcopy.c +++ b/drivers/infiniband/ulp/sdp/sdp_bcopy.c @@ -96,7 +96,7 @@ void sdp_remove_large_sock(struct sdp_sock *ssk) } } -/* Like tcp_fin */ +/* Like tcp_fin - called when SDP_MID_DISCONNECT is received */ static void sdp_fin(struct sock *sk) { sdp_dbg(sk, "%s\n", __func__); @@ -104,6 +104,42 @@ static void sdp_fin(struct sock *sk) sk->sk_shutdown |= RCV_SHUTDOWN; sock_set_flag(sk, SOCK_DONE); + switch (sk->sk_state) { + case TCP_SYN_RECV: + case TCP_ESTABLISHED: + sdp_set_state(sk, TCP_CLOSE_WAIT); + break; + + case TCP_FIN_WAIT1: + /* This case occurs when a simultaneous close + * happens, we must ack the received FIN and + * enter the CLOSING state. + */ + sdp_set_state(sk, TCP_CLOSING); + break; + + case TCP_FIN_WAIT2: + /* Received a reply FIN - start Infiniband tear down */ + sdp_set_state(sk, TCP_TIME_WAIT); + sdp_dbg(sk, "%s: Starting Infiniband tear down sending DREQ\n", + __func__); + + if (sdp_sk(sk)->id) { + rdma_disconnect(sdp_sk(sk)->id); + } else { + sdp_warn(sk, "%s: sdp_sk(sk)->id is NULL\n", __func__); + BUG(); + } + break; + case TCP_TIME_WAIT: + case TCP_CLOSE: + break; + default: + sdp_warn(sk, "%s: FIN in unexpected state. sk->sk_state=%d\n", + __func__, sk->sk_state); + break; + } + sk_mem_reclaim(sk); @@ -524,7 +560,9 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) if (unlikely(c < ssk->rx_head - ssk->rx_tail) && likely(ssk->bufs > 1) && - likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE)) { + likely(ssk->tx_head - ssk->tx_tail < SDP_TX_SIZE) && + likely((1 << ssk->isk.sk.sk_state) & + (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2))) { skb = sdp_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), GFP_KERNEL); @@ -533,20 +571,16 @@ void sdp_post_sends(struct sdp_sock *ssk, int nonagle) sdp_post_send(ssk, skb, SDP_MID_DATA); } - if (unlikely((1 << ssk->isk.sk.sk_state) & - (TCPF_FIN_WAIT1 | TCPF_LAST_ACK)) && + if (unlikely(ssk->sdp_disconnect) && !ssk->isk.sk.sk_send_head && ssk->bufs > (ssk->remote_credits >= ssk->rx_head - ssk->rx_tail)) { + ssk->sdp_disconnect = 0; skb = sdp_stream_alloc_skb(&ssk->isk.sk, sizeof(struct sdp_bsdh), gfp_page); /* FIXME */ BUG_ON(!skb); sdp_post_send(ssk, skb, SDP_MID_DISCONN); - if (ssk->isk.sk.sk_state == TCP_FIN_WAIT1) - sdp_set_state(&ssk->isk.sk, TCP_FIN_WAIT2); - else - sdp_set_state(&ssk->isk.sk, TCP_CLOSING); } } @@ -590,6 +624,7 @@ static void sdp_handle_resize_ack(struct sdp_sock *ssk, struct sdp_chrecvbuf *bu static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) { + struct sock *sk = &ssk->isk.sk; int frags; struct sk_buff *skb; struct sdp_bsdh *h; @@ -603,16 +638,15 @@ static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) if (unlikely(wc->status)) { if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, - "Recv completion with error. " - "Status %d\n", wc->status); - sdp_reset(&ssk->isk.sk); + sdp_dbg(sk, "Recv completion with error. Status %d\n", + wc->status); + sdp_reset(sk); } __kfree_skb(skb); return 0; } - sdp_dbg_data(&ssk->isk.sk, "Recv completion. ID %d Length %d\n", + sdp_dbg_data(sk, "Recv completion. ID %d Length %d\n", (int)wc->wr_id, wc->byte_len); if (unlikely(wc->byte_len < sizeof(struct sdp_bsdh))) { printk(KERN_WARNING "SDP BUG! byte_len %d < %zd\n", @@ -651,7 +685,7 @@ static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) } if (unlikely(h->flags & SDP_OOB_PEND)) - sk_send_sigurg(&ssk->isk.sk); + sk_send_sigurg(sk); skb_pull(skb, sizeof(struct sdp_bsdh)); @@ -661,21 +695,36 @@ static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) __kfree_skb(skb); break; } - skb = sdp_sock_queue_rcv_skb(&ssk->isk.sk, skb); + + if (unlikely(sk->sk_shutdown & RCV_SHUTDOWN)) { + /* got data in RCV_SHUTDOWN */ + if ((1 << sk->sk_state) & + (TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2)) { + /* go into abortive close */ + sdp_set_state(sk, TCP_TIME_WAIT); + + sk->sk_prot->disconnect(sk, 0); + } + + __kfree_skb(skb); + break; + } + skb = sdp_sock_queue_rcv_skb(sk, skb); if (unlikely(h->flags & SDP_OOB_PRES)) sdp_urg(ssk, skb); break; case SDP_MID_DISCONN: - /* this will wake recvmsg */ - sdp_sock_queue_rcv_skb(&ssk->isk.sk, skb); - sdp_fin(&ssk->isk.sk); + if (ssk->fin_wait_timeout) + sdp_cancel_fin_wait_timeout(ssk); + + __kfree_skb(skb); + sdp_fin(sk); break; case SDP_MID_CHRCVBUF: sdp_handle_resize_request(ssk, - (struct sdp_chrecvbuf *)skb->data); + (struct sdp_chrecvbuf *)skb->data); __kfree_skb(skb); break; - case SDP_MID_CHRCVBUF_ACK: sdp_handle_resize_ack(ssk, (struct sdp_chrecvbuf *)skb->data); __kfree_skb(skb); @@ -685,27 +734,62 @@ static int sdp_handle_recv_comp(struct sdp_sock *ssk, struct ib_wc *wc) printk(KERN_WARNING "SDP: FIXME MID %d\n", h->mid); __kfree_skb(skb); } + return 0; } static int sdp_handle_send_comp(struct sdp_sock *ssk, struct ib_wc *wc) { struct sk_buff *skb; + struct sdp_bsdh *h; skb = sdp_send_completion(ssk, wc->wr_id); if (unlikely(!skb)) return -1; - sk_wmem_free_skb(&ssk->isk.sk, skb); + if (unlikely(wc->status)) { if (wc->status != IB_WC_WR_FLUSH_ERR) { - sdp_dbg(&ssk->isk.sk, - "Send completion with error. " + sdp_dbg(&ssk->isk.sk, "Send completion with error. " "Status %d\n", wc->status); sdp_set_error(&ssk->isk.sk, -ECONNRESET); wake_up(&ssk->wq); } + goto out; } + h = (struct sdp_bsdh *)skb->data; + + if (likely(h->mid != SDP_MID_DISCONN)) + goto out; + + switch (ssk->isk.sk.sk_state) { + case TCP_FIN_WAIT1: + /* sdp_set_state(&ssk->isk.sk, + TCP_FIN_WAIT2); */ + break; + case TCP_CLOSING: + case TCP_LAST_ACK: + sdp_set_state(&ssk->isk.sk, TCP_TIME_WAIT); + + sdp_dbg(&ssk->isk.sk, + "%s: waiting for Infiniband tear down\n", __func__); + + /* We will not issue a DREQ here - because DREQ is sent on + the CM QP - and could reach the peer before the + SDP_MID_DISCONNECT This will put the peer in abortive close + state - without any real reason. + TODO: maybe a timeout whould be used here - what if the + peer won't send a DREQ? */ + break; + default: + sdp_dbg(&ssk->isk.sk, + "%s: sent DISCONNECT from unexpected state %d\n", + __func__, ssk->isk.sk.sk_state); + } + +out: + sk_wmem_free_skb(&ssk->isk.sk, skb); + return 0; } @@ -734,13 +818,6 @@ static void sdp_handle_wc(struct sdp_sock *ssk, struct ib_wc *wc) return; } - - if (ssk->time_wait && !ssk->isk.sk.sk_send_head && - ssk->tx_head == ssk->tx_tail) { - sdp_dbg(&ssk->isk.sk, "%s: destroy in time wait state\n", - __func__); - sdp_time_wait_destroy_sk(ssk); - } } void sdp_completion_handler(struct ib_cq *cq, void *cq_context) diff --git a/drivers/infiniband/ulp/sdp/sdp_cma.c b/drivers/infiniband/ulp/sdp/sdp_cma.c index f2fb083..8759bac 100644 --- a/drivers/infiniband/ulp/sdp/sdp_cma.c +++ b/drivers/infiniband/ulp/sdp/sdp_cma.c @@ -498,9 +498,17 @@ int sdp_cma_handler(struct rdma_cm_id *id, struct rdma_cm_event *event) ((struct sockaddr_in *)&id->route.addr.src_addr)->sin_addr.s_addr; rc = sdp_connected_handler(sk, event); break; - case RDMA_CM_EVENT_DISCONNECTED: + case RDMA_CM_EVENT_DISCONNECTED: /* This means DREQ/DREP received */ sdp_dbg(sk, "RDMA_CM_EVENT_DISCONNECTED\n"); rdma_disconnect(id); + + if (sk->sk_state != TCP_TIME_WAIT) { + sdp_set_error(sk, EPIPE); + rc = sdp_disconnected_handler(sk); + } + break; + case RDMA_CM_EVENT_TIMWAIT_EXIT: + sdp_dbg(sk, "RDMA_CM_EVENT_TIMEWAIT_EXIT\n"); rc = sdp_disconnected_handler(sk); break; case RDMA_CM_EVENT_DEVICE_REMOVAL: diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c index acd3aab..960ec90 100644 --- a/drivers/infiniband/ulp/sdp/sdp_main.c +++ b/drivers/infiniband/ulp/sdp/sdp_main.c @@ -342,10 +342,7 @@ void sdp_reset_sk(struct sock *sk, int rc) memset((void *)&ssk->id, 0, sizeof(*ssk) - offsetof(typeof(*ssk), id)); - if (ssk->time_wait) { - sdp_dbg(sk, "%s: destroy in time wait state\n", __func__); - sdp_time_wait_destroy_sk(ssk); - } + queue_work(sdp_workqueue, &ssk->destroy_work); sk->sk_state_change(sk); @@ -436,9 +433,10 @@ done: sdp_dbg(sk, "%s done\n", __func__); } -static void sdp_send_active_reset(struct sock *sk, gfp_t priority) +static void sdp_send_disconnect(struct sock *sk) { - sk->sk_prot->disconnect(sk, 0); + sdp_sk(sk)->sdp_disconnect = 1; + sdp_post_sends(sdp_sk(sk), 0); } /* @@ -452,7 +450,7 @@ static int sdp_close_state(struct sock *sk) return 0; if (sk->sk_state == TCP_ESTABLISHED) - sdp_set_state(sk, TCP_FIN_WAIT1); + sdp_set_state(sk, TCP_FIN_WAIT2);/* should be TCP_FIN_WAIT1 */ else if (sk->sk_state == TCP_CLOSE_WAIT) sdp_set_state(sk, TCP_LAST_ACK); else @@ -466,7 +464,13 @@ static void sdp_close(struct sock *sk, long timeout) struct sk_buff *skb; int data_was_unread = 0; + if ((1 << sk->sk_state) & (TCPF_TIME_WAIT | TCPF_CLOSE)) { + sock_put(sk); + return; + } + lock_sock(sk); + sock_hold(sk); sdp_dbg(sk, "%s\n", __func__); @@ -504,18 +508,23 @@ static void sdp_close(struct sock *sk, long timeout) if (data_was_unread) { /* Unread data was tossed, zap the connection. */ NET_INC_STATS_USER(LINUX_MIB_TCPABORTONCLOSE); - sdp_set_state(sk, TCP_CLOSE); - sdp_send_active_reset(sk, GFP_KERNEL); + sdp_set_state(sk, TCP_TIME_WAIT); + + /* Go into abortive close */ + sk->sk_prot->disconnect(sk, 0); } else if (sock_flag(sk, SOCK_LINGER) && !sk->sk_lingertime) { /* Check zero linger _after_ checking for unread data. */ - sk->sk_prot->disconnect(sk, 0); NET_INC_STATS_USER(LINUX_MIB_TCPABORTONDATA); + sdp_set_state(sk, TCP_TIME_WAIT); + + /* Go into abortive close */ + sk->sk_prot->disconnect(sk, 0); } else if (sdp_close_state(sk)) { /* We FIN if the application ate all the data before * zapping the connection. */ - sdp_post_sends(sdp_sk(sk), 0); + sdp_send_disconnect(sk); } /* TODO: state should move to CLOSE or CLOSE_WAIT etc on disconnect. @@ -533,7 +542,6 @@ adjudge_to_death: */ lock_sock(sk); - sock_hold(sk); sock_orphan(sk); /* This is a (useful) BSD violating of the RFC. There is a @@ -549,15 +557,8 @@ adjudge_to_death: * consume significant resources. Let's do it with special * linger2 option. --ANK */ - - if (sk->sk_state == TCP_FIN_WAIT2 && - !sk->sk_send_head && - sdp_sk(sk)->tx_head == sdp_sk(sk)->tx_tail) { - sdp_set_state(sk, TCP_CLOSE); - } - if ((1 << sk->sk_state) & (TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2)) { - sdp_sk(sk)->time_wait = 1; + sdp_sk(sk)->fin_wait_timeout = 1; /* TODO: liger2 unimplemented. We should wait 3.5 * rto. How do I know rto? */ /* TODO: tcp_fin_time to get timeout */ @@ -566,18 +567,20 @@ adjudge_to_death: atomic_inc(sk->sk_prot->orphan_count); queue_delayed_work(sdp_workqueue, &sdp_sk(sk)->time_wait_work, TCP_FIN_TIMEOUT); - goto out; } /* TODO: limit number of orphaned sockets. TCP has sysctl_tcp_mem and sysctl_tcp_max_orphans */ - sock_put(sk); - /* Otherwise, socket is reprieved until protocol close. */ -out: - sdp_dbg(sk, "%s: last socket put %d\n", __func__, - atomic_read(&sk->sk_refcnt)); + if (sk->sk_state != TCP_CLOSE) { + sdp_dbg(sk, "Hold socket till end of Infiniband tear-down\n"); + /* Hold socket till end of Infiniband tear-down */ + sock_hold(sk); + } + release_sock(sk); + + sock_put(sk); sk_common_release(sk); } @@ -635,11 +638,13 @@ static int sdp_disconnect(struct sock *sk, int flags) struct rdma_cm_id *id; sdp_dbg(sk, "%s\n", __func__); - if (ssk->id) - rc = rdma_disconnect(ssk->id); - if (old_state != TCP_LISTEN) + if (old_state != TCP_LISTEN) { + if (ssk->id) + rc = rdma_disconnect(ssk->id); + return rc; + } sdp_set_state(sk, TCP_CLOSE); id = ssk->id; @@ -827,14 +832,23 @@ static int sdp_ioctl(struct sock *sk, int cmd, unsigned long arg) return put_user(answ, (int __user *)arg); } +void sdp_cancel_fin_wait_timeout(struct sdp_sock *ssk) +{ + ssk->fin_wait_timeout = 0; + cancel_delayed_work(&ssk->time_wait_work); + atomic_dec(ssk->isk.sk.sk_prot->orphan_count); +} + void sdp_destroy_work(struct work_struct *work) { struct sdp_sock *ssk = container_of(work, struct sdp_sock, destroy_work); struct sock *sk = &ssk->isk.sk; sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - cancel_delayed_work(&sdp_sk(sk)->time_wait_work); - atomic_dec(sk->sk_prot->orphan_count); + if (ssk->fin_wait_timeout) + sdp_cancel_fin_wait_timeout(ssk); + + sdp_set_state(sk, TCP_CLOSE); sock_put(sk); } @@ -846,26 +860,20 @@ void sdp_time_wait_work(struct work_struct *work) lock_sock(sk); sdp_dbg(sk, "%s\n", __func__); - if (!sdp_sk(sk)->time_wait) { + if (!sdp_sk(sk)->fin_wait_timeout) { release_sock(sk); return; } sdp_dbg(sk, "%s: refcnt %d\n", __func__, atomic_read(&sk->sk_refcnt)); - sdp_set_state(sk, TCP_CLOSE); - sdp_sk(sk)->time_wait = 0; + sdp_sk(sk)->fin_wait_timeout = 0; release_sock(sk); atomic_dec(sk->sk_prot->orphan_count); - sock_put(sk); -} -void sdp_time_wait_destroy_sk(struct sdp_sock *ssk) -{ - ssk->time_wait = 0; - sdp_set_state(&ssk->isk.sk, TCP_CLOSE); - queue_work(sdp_workqueue, &ssk->destroy_work); + if (sdp_sk(sk)->id) + rdma_disconnect(sdp_sk(sk)->id); } static int sdp_init_sock(struct sock *sk) @@ -880,6 +888,9 @@ static int sdp_init_sock(struct sock *sk) INIT_WORK(&ssk->destroy_work, sdp_destroy_work); sk->sk_route_caps |= NETIF_F_SG | NETIF_F_NO_CSUM; + + ssk->sdp_disconnect = 0; + return 0; } @@ -895,7 +906,7 @@ static void sdp_shutdown(struct sock *sk, int how) return; if (sk->sk_state == TCP_ESTABLISHED) - sdp_set_state(sk, TCP_FIN_WAIT1); + sdp_set_state(sk, TCP_FIN_WAIT2);/* should be TCP_FIN_WAIT1 */ else if (sk->sk_state == TCP_CLOSE_WAIT) sdp_set_state(sk, TCP_LAST_ACK); else @@ -910,7 +921,7 @@ static void sdp_shutdown(struct sock *sk, int how) if (ssk->nonagle & TCP_NAGLE_OFF) ssk->nonagle |= TCP_NAGLE_PUSH; - sdp_post_sends(ssk, 0); + sdp_send_disconnect(sk); } static void sdp_mark_push(struct sdp_sock *ssk, struct sk_buff *skb) -- 1.5.3 From jackm at dev.mellanox.co.il Thu Jul 10 01:10:56 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 11:10:56 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <3b5e77ad0807100047u1790dd7chf5b59f14be0684a4@mail.gmail.com> References: <200807101039.41832.jackm@dev.mellanox.co.il> <3b5e77ad0807100047u1790dd7chf5b59f14be0684a4@mail.gmail.com> Message-ID: <200807101110.56602.jackm@dev.mellanox.co.il> On Thursday 10 July 2008 10:47, Ron Livne wrote: > > I'm going to release some patches, based on your XRC patches, in which > I'm going to replace xrc_ops with more_ops. > These patches will will be relative to your patches. > I hope it's ok with you. > > Ron > It is OK with me. I will make the name change (xrc_ops to more_ops) already in the updated patch set I'll be submitting later today to the list. You can then add your changes on top of mine. - Jack From ronli at voltaire.com Thu Jul 10 04:15:22 2008 From: ronli at voltaire.com (Ron Livne) Date: Thu, 10 Jul 2008 11:15:22 +0000 (UTC) Subject: [ofa-general] Problem building libmlx4 Message-ID: Hi, I pulled libmlx4 from: git://git.kernel.org/pub/scm/libs/infiniband/libmlx4.git I tried building it with: ./autogen.sh ./configure but the configure script renders the following error: checking size of long... configure: error: cannot compute sizeof (long), 77 I'm using Redhat 5.1 with kernel 2.6.26-rc2 on an x86-64 machine. Do you have an idea for solving this? Thank you, Ron From mdidomenico4 at gmail.com Thu Jul 10 01:30:11 2008 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Thu, 10 Jul 2008 04:30:11 -0400 Subject: [ofa-general] IPoIB arps disappearing Message-ID: I'm having a bit of a weird problem that i cannot figure out. If anyone can help from the community it would be appreciated. Here's the packet flow cn(ib0)->io(ib0)->io(eth5)->pan(*) cn = compute node io = io node pan = panasas storage network We have 12 shelves of panasas network storage on a seperate network, which is being fronted by bridge servers which are routing IPoIB traffic to 10G ethernet traffic. We're using Mellanox Connect-X Ethernet/IB adapters everwhere. We're running Ofed 1.3.1 and the latest firmwares for IB/Eth everywhere. Here's the problem. I can mount the storage on the compute nodes, but if i try to send anything more then 50MB of data via dd. I seem to loose the ARP entries for the compute nodes on the IO servers. This seems to happen whether I use the filesystem or a netperf run from the compute node to the panasas storage I can run netperf between the compute node and io node and get full IPoIB line rate with no issues I can run netperf between the io node and the panasas storage and get full 10G ethernet line rate with no issues When looking at the TCP traces, i can clearly see that a big chunk of data is sent between the end-points and then it stalls. Immediately after the stall is an ARP request and then another chunk of data, and this scenario repeats over and over. Any thoughts or questions? Thanks - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Thu Jul 10 02:41:30 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Thu, 10 Jul 2008 02:41:30 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080710-0200 daily build status Message-ID: <20080710094131.41504E6096A@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080710-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Thu Jul 10 02:56:28 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 10 Jul 2008 12:56:28 +0300 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: References: <487500A9.5050707@opengridcomputing.com> Message-ID: <4875DCCC.7020404@voltaire.com> Roland Dreier wrote: > Would have been nice for the original author of a patch changing locking to test with lockdep (IB needed the mutex_lock_nested treatment too), but oh well. Yes, sounds like testing the original patch with lockdep would have bumped something earlier, thanks for fixing that. > yep, the mutex_lock for conn_id in iw_conn_req_handler should really be > > mutex_lock_nested(&conn_id->handler_mutex, SINGLE_DEPTH_NESTING); > > since it's already inside another lock of the same class. but I don't > understand why it leads to a real deadlock... they're different mutexes > and I don't see how the conn_id lock could already be locked?? I am not sure to follow what does "class" means in this context, is this struct rdma_id_private? such that "inside another lock of the same class" means locking the same mutex for another instance of this structure? Or. From eli at dev.mellanox.co.il Thu Jul 10 04:01:57 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 10 Jul 2008 14:01:57 +0300 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: References: Message-ID: <20080710110157.GA4004@mtls03> On Thu, Jul 10, 2008 at 04:30:11AM -0400, Michael Di Domenico wrote: > I'm having a bit of a weird problem that i cannot figure out. If anyone can > help from the community it would be appreciated. > Here's the packet flow > > cn(ib0)->io(ib0)->io(eth5)->pan(*) > > cn = compute node > io = io node > pan = panasas storage network > > We have 12 shelves of panasas network storage on a seperate network, which > is being fronted by bridge servers which are routing IPoIB traffic to 10G > ethernet traffic. We're using Mellanox Connect-X Ethernet/IB adapters > everwhere. We're running Ofed 1.3.1 and the latest firmwares for IB/Eth > everywhere. > > Here's the problem. I can mount the storage on the compute nodes, but if i > try to send anything more then 50MB of data via dd. I seem to loose the ARP > entries for the compute nodes on the IO servers. This seems to happen > whether I use the filesystem or a netperf run from the compute node to the > panasas storage > > I can run netperf between the compute node and io node and get full IPoIB > line rate with no issues > I can run netperf between the io node and the panasas storage and get full > 10G ethernet line rate with no issues > > When looking at the TCP traces, i can clearly see that a big chunk of data > is sent between the end-points and then it stalls. Immediately after the > stall is an ARP request and then another chunk of data, and this scenario > repeats over and over. > > Any thoughts or questions? > Michael, could you repeat the experiment with debugging enabled? For IPoIB, this can be done as follows: echo 1 > /sys/module/ib_ipoib/parameters/debug_level echo 1 > /sys/module/ib_ipoib/parameters/macast_debug_level Please send the output of dmesg after the failure. Thanks. From tziporet at dev.mellanox.co.il Thu Jul 10 04:03:12 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 10 Jul 2008 14:03:12 +0300 Subject: [ofa-general] Re: [ewg] Developer's Workshop at SC'08 In-Reply-To: References: Message-ID: <4875EC70.1070605@mellanox.co.il> Woodruff, Robert J wrote: > > The question is, do you think we need a developers workshop that week as > well to perhaps plan future work, OFED 1.5, WinOF 2.1 etc. or do you think a > face-to-face meeting is not needed this fall and wait till next spring > at Sonoma for the next face-to-face. > > > I think one face-2-face meeting in Sonoma is enough at this point. Regarding OFED coordination most of the work is done in the bi-weekly meetings. Of course if many of us will come to SC08 we can set a meting between us but this does not require full workshop organization. Tziporet From mdidomenico4 at gmail.com Thu Jul 10 04:46:50 2008 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Thu, 10 Jul 2008 07:46:50 -0400 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: <20080710110157.GA4004@mtls03> References: <20080710110157.GA4004@mtls03> Message-ID: Do you want the debug from the IO, Compute, or Both? On Thu, Jul 10, 2008 at 7:01 AM, Eli Cohen wrote: > On Thu, Jul 10, 2008 at 04:30:11AM -0400, Michael Di Domenico wrote: > > I'm having a bit of a weird problem that i cannot figure out. If anyone > can > > help from the community it would be appreciated. > > Here's the packet flow > > > > cn(ib0)->io(ib0)->io(eth5)->pan(*) > > > > cn = compute node > > io = io node > > pan = panasas storage network > > > > We have 12 shelves of panasas network storage on a seperate network, > which > > is being fronted by bridge servers which are routing IPoIB traffic to 10G > > ethernet traffic. We're using Mellanox Connect-X Ethernet/IB adapters > > everwhere. We're running Ofed 1.3.1 and the latest firmwares for IB/Eth > > everywhere. > > > > Here's the problem. I can mount the storage on the compute nodes, but if > i > > try to send anything more then 50MB of data via dd. I seem to loose the > ARP > > entries for the compute nodes on the IO servers. This seems to happen > > whether I use the filesystem or a netperf run from the compute node to > the > > panasas storage > > > > I can run netperf between the compute node and io node and get full IPoIB > > line rate with no issues > > I can run netperf between the io node and the panasas storage and get > full > > 10G ethernet line rate with no issues > > > > When looking at the TCP traces, i can clearly see that a big chunk of > data > > is sent between the end-points and then it stalls. Immediately after the > > stall is an ARP request and then another chunk of data, and this scenario > > repeats over and over. > > > > Any thoughts or questions? > > > > Michael, > could you repeat the experiment with debugging enabled? For IPoIB, this > can be done as follows: > > echo 1 > /sys/module/ib_ipoib/parameters/debug_level > echo 1 > /sys/module/ib_ipoib/parameters/macast_debug_level > > Please send the output of dmesg after the failure. > > Thanks. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdidomenico4 at gmail.com Thu Jul 10 04:53:16 2008 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Thu, 10 Jul 2008 07:53:16 -0400 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: References: <20080710110157.GA4004@mtls03> Message-ID: Turning on this debug gives this error [root at cfd-io-0001 ~]# echo 1 > /sys/module/ib_ipoib/parameters/macast_debug_level -bash: /sys/module/ib_ipoib/parameters/macast_debug_level: Permission denied dmesg output after i start the netperf test (which doesn't complete) mtnic 0000:02:00.0: Port 2 - link up mtnic 0000:02:00.0: Port 2 - link down mtnic 0000:02:00.0: Freed 1 uncompleted tx descriptors mtnic 0000:02:00.0: Port 2 - link up ib0: mtu > 2044 will cause multicast packet drops. eth5: no IPv6 routers present ib0: Send unicast ARP to 0384 ib0: REQ arrived ib0: Request connection 0x2c0406 for gid fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 ib0: REP received. ib0: Send unicast ARP to 0384 On Thu, Jul 10, 2008 at 7:46 AM, Michael Di Domenico wrote: > Do you want the debug from the IO, Compute, or Both? > > > On Thu, Jul 10, 2008 at 7:01 AM, Eli Cohen wrote: > >> On Thu, Jul 10, 2008 at 04:30:11AM -0400, Michael Di Domenico wrote: >> > I'm having a bit of a weird problem that i cannot figure out. If anyone >> can >> > help from the community it would be appreciated. >> > Here's the packet flow >> > >> > cn(ib0)->io(ib0)->io(eth5)->pan(*) >> > >> > cn = compute node >> > io = io node >> > pan = panasas storage network >> > >> > We have 12 shelves of panasas network storage on a seperate network, >> which >> > is being fronted by bridge servers which are routing IPoIB traffic to >> 10G >> > ethernet traffic. We're using Mellanox Connect-X Ethernet/IB adapters >> > everwhere. We're running Ofed 1.3.1 and the latest firmwares for IB/Eth >> > everywhere. >> > >> > Here's the problem. I can mount the storage on the compute nodes, but >> if i >> > try to send anything more then 50MB of data via dd. I seem to loose the >> ARP >> > entries for the compute nodes on the IO servers. This seems to happen >> > whether I use the filesystem or a netperf run from the compute node to >> the >> > panasas storage >> > >> > I can run netperf between the compute node and io node and get full >> IPoIB >> > line rate with no issues >> > I can run netperf between the io node and the panasas storage and get >> full >> > 10G ethernet line rate with no issues >> > >> > When looking at the TCP traces, i can clearly see that a big chunk of >> data >> > is sent between the end-points and then it stalls. Immediately after >> the >> > stall is an ARP request and then another chunk of data, and this >> scenario >> > repeats over and over. >> > >> > Any thoughts or questions? >> > >> >> Michael, >> could you repeat the experiment with debugging enabled? For IPoIB, this >> can be done as follows: >> >> echo 1 > /sys/module/ib_ipoib/parameters/debug_level >> echo 1 > /sys/module/ib_ipoib/parameters/macast_debug_level >> >> Please send the output of dmesg after the failure. >> >> Thanks. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From John.Marshall at ec.gc.ca Thu Jul 10 05:07:39 2008 From: John.Marshall at ec.gc.ca (John Marshall) Date: Thu, 10 Jul 2008 08:07:39 -0400 Subject: [ofa-general] debian build instructions? In-Reply-To: <487281C6.5070001@ec.gc.ca> References: <487281C6.5070001@ec.gc.ca> Message-ID: <4875FB8B.4020507@ec.gc.ca> John Marshall wrote: > Hi, > > I have muddled my way through building the latest > ofed-1.3.1 on debian etch. Is there a simple way of > doing this that I am missing? I have searched for info > and found some, but nothing that gives a step by > step procedure. There seem to be a lot of people on this list. Is there no one working with debian that might be able to give a pointer? For example, how did you go about building from the source? Is it best to start with the RPMs or the git repository? Did you run alien on the RPMs, untar, autogen.sh, ./configure, and massage a few things by hand? John From ronli.voltaire at gmail.com Thu Jul 10 05:21:06 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Thu, 10 Jul 2008 15:21:06 +0300 Subject: [ofa-general] Problem building libmlx4 In-Reply-To: References: Message-ID: <3b5e77ad0807100521y3e1d0f5w3e97be8dfda64430@mail.gmail.com> I edited the configure script, canceled the sizeof long test, and manually entered there that the size is 8. everything seems to be working fine now. Ron On Thu, Jul 10, 2008 at 2:15 PM, Ron Livne wrote: > Hi, > > I pulled libmlx4 from: > git://git.kernel.org/pub/scm/libs/infiniband/libmlx4.git > > I tried building it with: > ./autogen.sh > ./configure > > > but the configure script renders the following error: > checking size of long... configure: error: cannot compute sizeof (long), > 77 > > I'm using Redhat 5.1 with kernel 2.6.26-rc2 on an x86-64 machine. > > Do you have an idea for solving this? > > Thank you, > Ron > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From jackm at dev.mellanox.co.il Thu Jul 10 05:22:53 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 15:22:53 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <3b5e77ad0807100047u1790dd7chf5b59f14be0684a4@mail.gmail.com> References: <200807101039.41832.jackm@dev.mellanox.co.il> <3b5e77ad0807100047u1790dd7chf5b59f14be0684a4@mail.gmail.com> Message-ID: <200807101522.54182.jackm@dev.mellanox.co.il> On Thursday 10 July 2008 10:47, Ron Livne wrote: > >can we do this so that the new verb lies after XRC in ib_context, since OFED 1.3 went out > >with XRC in userspace (so we don't break binary compatibility): > > > >struct ibv_context { > > struct ibv_device *device; > > struct ibv_context_ops ops; > > int cmd_fd; > > int async_fd; > > int num_comp_vectors; > > pthread_mutex_t mutex; > > void *abi_compat; > > struct ibv_xrc_ops *xrc_ops; > > /* ==> new verb ops structure pointer here */ > >}; > > I'm going to release some patches, based on your XRC patches, in which > I'm going to replace xrc_ops with more_ops. > These patches will will be relative to your patches. > I hope it's ok with you. > Roland, I think we can do things something like I have below (only partial -- I've not changed all usage of xrc_ops in ibv_context -- I'll do this in my patches). Regarding undefining of HAVE_IBV_XRC_OPS, I've not figured out a way of conditionally invoking AC_CHECK_HEADER if the MORE_OPS structure is not present. Please note that using AC_CHECK_HEADER has the advantage of applying to the module being built (I don't need to do anything special when configuring libibverbs). If you can see a cleaner way than this to add extra ops (e.g., conditionally structuring the AC_CHECK_HEADER calls), please let me know. - Jack *** IN LIBIBVERBS *** diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index 036a0c5..2865d5c 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -645,7 +645,7 @@ struct ibv_device { char ibdev_path[IBV_SYSFS_PATH_MAX]; }; -struct ibv_xrc_ops { +struct ibv_more_ops { struct ibv_srq * (*create_xrc_srq)(struct ibv_pd *pd, struct ibv_xrc_domain *xrc_domain, struct ibv_cq *xrc_cq, @@ -737,7 +737,7 @@ struct ibv_context { int num_comp_vectors; pthread_mutex_t mutex; void *abi_compat; - struct ibv_xrc_ops *xrc_ops; + struct ibv_more_ops *more_ops; }; *** IN LIBMLX4 *** diff --git a/configure.in b/configure.in index 9304539..8db22ab 100644 --- a/configure.in +++ b/configure.in @@ -42,8 +42,14 @@ AC_CHECK_HEADER(valgrind/memcheck.h, dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_CHECK_SIZEOF(long) -AC_CHECK_MEMBER(struct ibv_context.xrc_ops, - [AC_DEFINE([HAVE_IBV_XRC_OPS], 1, [Define to 1 if xrc_ops is a member of ibv_context])],, +AC_CHECK_MEMBER(struct ibv_context.more_ops, + [AC_DEFINE([HAVE_IBV_MORE_OPS], 1, [Define to 1 if more_ops is a member of ibv_context])],, + [#include ]) +AC_CHECK_MEMBER(struct ibv_more_ops.create_xrc_srq, + [AC_DEFINE([HAVE_IBV_XRC_OPS], 1, [Define to 1 if have xrc ops])],, + [#include ]) +AC_CHECK_MEMBER(struct ibv_more_ops.create_qp_expanded, + [AC_DEFINE([HAVE_IBV_CREATE_QP_EX], 1, [Define to 1 if have create_qp_expanded])],, [#include ]) dnl Checks for library functions diff --git a/src/mlx4.c b/src/mlx4.c index 18333e7..b6970a0 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -68,8 +68,9 @@ struct { HCA(MELLANOX, 0x673c), /* MT25408 "Hermon" QDR PCIe gen2 */ }; -#ifdef HAVE_IBV_XRC_OPS +#ifdef HAVE_IBV_MORE_OPS static struct ibv_xrc_ops mlx4_xrc_ops = { +#ifdef HAVE_IBV_XRC_OPS .create_xrc_srq = mlx4_create_xrc_srq, .open_xrc_domain = mlx4_open_xrc_domain, .close_xrc_domain = mlx4_close_xrc_domain, @@ -78,6 +79,10 @@ static struct ibv_xrc_ops mlx4_xrc_ops = { .query_xrc_rcv_qp = mlx4_query_xrc_rcv_qp, .reg_xrc_rcv_qp = mlx4_reg_xrc_rcv_qp, .unreg_xrc_rcv_qp = mlx4_unreg_xrc_rcv_qp, +#endif +#ifdef HAVE_IBV_CREATE_QP_EXP + **** Ron Livne's stuff here +#endif }; #endif @@ -177,7 +182,7 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ pthread_spin_init(&context->uar_lock, PTHREAD_PROCESS_PRIVATE); context->ibv_ctx.ops = mlx4_ctx_ops; -#ifdef HAVE_IBV_XRC_OPS +#if defined(HAVE_IBV_XRC_OPS) || defined(HAVE_IBV_CREATE_QP_EXP) context->ibv_ctx.xrc_ops = &mlx4_xrc_ops; #endif diff --git a/src/mlx4.h b/src/mlx4.h index a791901..a2e3aec 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -79,6 +79,11 @@ #endif +#ifndef HAVE_IBV_MORE_OPS +#undef HAVE_IBV_XRC_OPS +#undef HAVE_IBV_CREATE_QP_EXP +#endif + #define HIDDEN __attribute__((visibility ("hidden"))) #define PFX "mlx4: " From eli at dev.mellanox.co.il Thu Jul 10 05:33:04 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 10 Jul 2008 15:33:04 +0300 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: References: <20080710110157.GA4004@mtls03> Message-ID: <20080710123304.GD4004@mtls03> On Thu, Jul 10, 2008 at 07:57:30AM -0400, Michael Di Domenico wrote: > maybe i spoke too soon, so more output came, i thought it was done > ib0: mtu > 2044 will cause multicast packet drops. > eth5: no IPv6 routers present > ib0: Send unicast ARP to 0384 > ib0: REQ arrived > ib0: Request connection 0x2c0406 for gid > fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 > ib0: REP received. > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 045e > ib0: REQ arrived > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 045e > ib0: REQ arrived > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d > ib0: Destroy active connection 0x2c0406 head 0x22644 tail 0x22644 > ib0: Request connection 0x2f0406 for gid > fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 > ib0: REP received. > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 > ib0: Destroy active connection 0x2f0406 head 0x4 tail 0x4 I see you're working in connected mode. Can you please do the follwoing: 1. clear dmesg: dmesg -c 2. run again, then send all the output of dmesg Do this for both connected and datagram modes. Thanks. From hrosenstock at xsigo.com Thu Jul 10 05:42:25 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 05:42:25 -0700 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080709211756.GS12551@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> Message-ID: <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-07-10 at 00:17 +0300, Sasha Khapyorsky wrote: > On 13:53 Wed 09 Jul , Hal Rosenstock wrote: > > > > > > It is just *.ver file, *.map is symbol versioning - there were additions, > > > no changes in existing functions API. > > > > Isn't that needed too when APIs are added ? > > I don't think. > > > Also, some existing > > functions had minor parameter changes (like const being added). > > Yes, but it does not touch ABI and doesn't affect a linker. > > > > > > this will take effect in the next daily build. > > > > > > > > The tar balls just issued won't be reissued ? > > > > > > There were some commits after this already. > > > > Shouldn't there be a branch for this if that is an issue ? > > And then to do libibmad-1.2.1.1? Looks like an overkill for this > particular case - 1.2.* is in development period and the issue is pretty > minor - we will fix it in today's daily build and in the next tarball > release. Also, what about backward compatibility with apps built with the previous version of these libraries ? That's mainly a libibmad issue right now. I'm pretty sure there are some out of tree apps using libibmad based on some of the previous email traffic on this list. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Thu Jul 10 06:05:11 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 06:05:11 -0700 Subject: [ofa-general] Issues with osm_lin_fwd_tbl.h:osm_lin_fwd_tbl_get_lids_per_block() In-Reply-To: <4874BF35.4060805@bull.net> References: <4874BF35.4060805@bull.net> Message-ID: <1215695111.18371.155.camel@hrosenstock-ws.xsigo.com> Hi Vincent, On Wed, 2008-07-09 at 15:37 +0200, Vincent Ficet wrote: > Hello, > > The function osm_lin_fwd_tbl.h:osm_lin_fwd_tbl_get_lids_per_block() is > coded as follows: > > static inline uint16_t > osm_lin_fwd_tbl_get_lids_per_block(IN const osm_lin_fwd_tbl_t * const p_tbl) > { > UNUSED_PARAM(p_tbl); > return (64); > } > > Given that the block used comes from the 'data' (payload) field of the > ib_smp_t structure (defined in opensm/include/iba/ib_types.h) which is: > > uint8_t data[IB_SMP_DATA_SIZE]; > > where IB_SMP_DATA_SIZE is defined as: > > #define IB_SMP_DATA_SIZE 64 > > Shouldn't osm_lin_fwd_tbl_get_lids_per_block() be implemented as follows: > > { > UNUSED_PARAM(p_tbl); > return IB_SMP_DATA_SIZE; > } It is bound by the SMP MAD data size but is actually defined in the LFT SM attribute. > Also, a LID beeing 16 bit wide, the function name is rather misleading > if not wrong: > In actual fact, it should rather be called > osm_lin_fwd_tbl_get_ports_per_block(), since output ports are encoded as > uint8_t in osm_lin_fwd_tbl.h: > > typedef struct osm_lin_fwdbl { > uint16_t size; > uint8_t port_tbl[1]; > } osm_lin_fwd_tbl_t; > > I think the confusion here is the same as a previous issue I raised in > commit a4934944004586759b3519afa71db5fddebb6541 > Do you agree ? If so, I can post a patch if you wish. LIDs are the input to the table lookup and port number is the output. I'm not sure which is better/less confusing to use in the API name. > Finally, what is the point in keeping the unused parameter p_tbl ? Yes, passing that parameter doesn't appear to be needed and could be eliminated. In fact, that whole routine could be eliminated and replaced by a define IMO and then have that ripple back into/simplify any routines which use it. Just my $0.02 on this. -- Hal > Thanks for your help, > > Vincent > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Thu Jul 10 06:05:18 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 06:05:18 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] opensm/osm_lin_fwd_tbl.h: Cosmetic changes Message-ID: <1215695118.18371.157.camel@hrosenstock-ws.xsigo.com> opensm/osm_lin_fwd_tbl.h: Cosmetic changes Signed-off-by: Hal Rosenstock diff --git a/opensm/include/opensm/osm_lin_fwd_tbl.h b/opensm/include/opensm/osm_lin_fwd_tbl.h index 98c6b4f..be3a3ee 100644 --- a/opensm/include/opensm/osm_lin_fwd_tbl.h +++ b/opensm/include/opensm/osm_lin_fwd_tbl.h @@ -84,7 +84,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct osm_lin_fwdbl { +typedef struct osm_lin_fwd_tbl { uint16_t size; uint8_t port_tbl[1]; } osm_lin_fwd_tbl_t; @@ -169,7 +169,6 @@ osm_lin_fwd_tbl_set(IN osm_lin_fwd_tbl_t * const p_tbl, if (lid_ho < p_tbl->size) p_tbl->port_tbl[lid_ho] = port; } - /* * PARAMETERS * p_tbl @@ -207,7 +206,6 @@ osm_lin_fwd_tbl_get(IN const osm_lin_fwd_tbl_t * const p_tbl, else return (OSM_NO_PATH); } - /* * PARAMETERS * p_tbl @@ -238,7 +236,6 @@ osm_lin_fwd_tbl_get_size(IN const osm_lin_fwd_tbl_t * const p_tbl) { return (p_tbl->size); } - /* * PARAMETERS * p_tbl @@ -267,7 +264,6 @@ osm_lin_fwd_tbl_get_lids_per_block(IN const osm_lin_fwd_tbl_t * const p_tbl) UNUSED_PARAM(p_tbl); return (64); } - /* * PARAMETERS * p_tbl @@ -298,7 +294,6 @@ osm_lin_fwd_tbl_get_max_block_id_in_use(IN const osm_lin_fwd_tbl_t * return ((uint16_t) (lid_top_ho / osm_lin_fwd_tbl_get_lids_per_block(p_tbl))); } - /* * PARAMETERS * p_tbl @@ -341,7 +336,6 @@ osm_lin_fwd_tbl_set_block(IN osm_lin_fwd_tbl_t * const p_tbl, memcpy(&p_tbl->port_tbl[lid_start], p_block, num_lids); return (IB_SUCCESS); } - /* * PARAMETERS * p_tbl From mdidomenico4 at gmail.com Thu Jul 10 06:21:34 2008 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Thu, 10 Jul 2008 09:21:34 -0400 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: <20080710123304.GD4004@mtls03> References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> Message-ID: tests using datagram IPoIB (non-connected mode) dmesg from the compute node cfd-cnsl-0001:~ # dmesg ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:007d MTU > 1024 ib0: PathRec LID 0x0161 for GID fe80:0000:0000:0000:00e0:8111:0100:007d ib0: Created ah ffff81042063dc80 ib0: created address handle ffff8102206144c0 for LID 0x0161, SL 0 ib0: Send unicast ARP to 0161 ib0: Send unicast ARP to 0161 ib0: Send unicast ARP to 0161 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0161 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000404 fe80:0000:0000:0000:00e0:8111:0100:0091 dmesg from the IO node [root at cfd-io-0001 ~]# dmesg ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib_mthca 0000:07:00.0: too many gathers ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib0: Send unicast ARP to 0384 ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 045e ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib0: Send unicast ARP to 0384 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: REQ arrived ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib_mthca 0000:07:00.0: opcode invalid ib0: post_send failed ib0: Send unicast ARP to 0384 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Send unicast ARP to 045e ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 On Thu, Jul 10, 2008 at 8:33 AM, Eli Cohen wrote: > On Thu, Jul 10, 2008 at 07:57:30AM -0400, Michael Di Domenico wrote: > > maybe i spoke too soon, so more output came, i thought it was done > > ib0: mtu > 2044 will cause multicast packet drops. > > eth5: no IPv6 routers present > > ib0: Send unicast ARP to 0384 > > ib0: REQ arrived > > ib0: Request connection 0x2c0406 for gid > > fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 > > ib0: REP received. > > ib0: Send unicast ARP to 0384 > > ib0: Send unicast ARP to 045e > > ib0: REQ arrived > > ib0: Send unicast ARP to 0384 > > ib0: Send unicast ARP to 0384 > > ib0: Send unicast ARP to 0384 > > ib0: Send unicast ARP to 045e > > ib0: REQ arrived > > ib0: neigh_destructor for bonding device: ib0 > > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d > > ib0: Destroy active connection 0x2c0406 head 0x22644 tail 0x22644 > > ib0: Request connection 0x2f0406 for gid > > fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 > > ib0: REP received. > > ib0: neigh_destructor for bonding device: ib0 > > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 > > ib0: Destroy active connection 0x2f0406 head 0x4 tail 0x4 > > > I see you're working in connected mode. Can you please do the > follwoing: > > 1. clear dmesg: dmesg -c > 2. run again, then send all the output of dmesg > > Do this for both connected and datagram modes. > > Thanks. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Thu Jul 10 06:17:24 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 06:17:24 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] opensm/osm_rand_fwd_tbl.h: Cosmetic changes Message-ID: <1215695844.18371.160.camel@hrosenstock-ws.xsigo.com> opensm/osm_rand_fwd_tbl.h: Cosmetic changes Signed-off-by: Hal Rosenstock diff --git a/opensm/include/opensm/osm_rand_fwd_tbl.h b/opensm/include/opensm/osm_rand_fwd_tbl.h index 8e9ac90..31098b9 100644 --- a/opensm/include/opensm/osm_rand_fwd_tbl.h +++ b/opensm/include/opensm/osm_rand_fwd_tbl.h @@ -35,8 +35,8 @@ /* * Abstract: - * Declaration of osm_switch_t. - * This object represents an IBA switch. + * Declaration of osm_rand_fwd_tbl_t. + * This object represents a random forwarding table. * This object is part of the OpenSM family of objects. */ @@ -87,7 +87,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct osm_rand_fwdbl { +typedef struct osm_rand_fwd_tbl { /* PLACE HOLDER STRUCTURE ONLY!! */ uint32_t size; } osm_rand_fwd_tbl_t; @@ -116,14 +116,13 @@ static inline void osm_rand_tbl_delete(IN osm_rand_fwd_tbl_t ** const pp_tbl) free(*pp_tbl); *pp_tbl = NULL; } - /* * PARAMETERS * pp_tbl * [in] Pointer a Pointer to the Random Forwarding Table object. * * RETURN VALUE -* On success, returns a pointer to a new Linear Forwarding Table object +* On success, returns a pointer to a new Random Forwarding Table object * of the specified size. * NULL otherwise. * @@ -151,7 +150,6 @@ osm_rand_fwd_tbl_set(IN osm_rand_fwd_tbl_t * const p_tbl, UNUSED_PARAM(port); CL_ASSERT(FALSE); } - /* * PARAMETERS * p_tbl @@ -192,7 +190,6 @@ osm_rand_fwd_tbl_set_block(IN osm_rand_fwd_tbl_t * const p_tbl, CL_ASSERT(FALSE); return (IB_ERROR); } - /* * PARAMETERS * p_tbl @@ -231,11 +228,10 @@ osm_rand_fwd_tbl_get(IN const osm_rand_fwd_tbl_t * const p_tbl, return (OSM_NO_PATH); } - /* * PARAMETERS * p_tbl -* [in] Pointer to the Linear Forwarding Table object. +* [in] Pointer to the Random Forwarding Table object. * * lid_ho * [in] LID value (host order) for which to get the route. @@ -263,7 +259,6 @@ osm_rand_fwd_tbl_get_lids_per_block(IN const osm_rand_fwd_tbl_t * const p_tbl) UNUSED_PARAM(p_tbl); return (16); } - /* * PARAMETERS * p_tbl @@ -296,7 +291,6 @@ osm_rand_fwd_tbl_get_max_block_id_in_use(IN const osm_rand_fwd_tbl_t * CL_ASSERT(FALSE); return (0); } - /* * PARAMETERS * p_tbl @@ -326,7 +320,6 @@ osm_rand_fwd_tbl_get_size(IN const osm_rand_fwd_tbl_t * const p_tbl) CL_ASSERT(FALSE); return (0); } - /* * PARAMETERS * p_tbl From wangwhao at cn.ibm.com Thu Jul 10 04:43:18 2008 From: wangwhao at cn.ibm.com (Wen Hao Wang) Date: Thu, 10 Jul 2008 19:43:18 +0800 Subject: [ofa-general] ***SPAM*** osmtest fails to create inventory file Message-ID: Hello all: I am using IBM HS21XM blades with 4X InfiniBand DDR Expansion Card, on x86_64 RHEL5.2 Server. opensm-3.1.8-1.el5 is installed. The IB device has worked for weeks. Today I tried "osmtest -f c" to generate inventory file. But it gave error 5501, 0003 and 0064. Would anyone give some advices? Thanks! [root at gaia-08 ~]# osmtest -f c Command Line Arguments Done with args Flow = Create Inventory Jul 10 13:23:45 588714 [F6AF1030] 0x7f -> Setting log level to: 0x03 Jul 10 13:23:45 606560 [F6AF1030] 0x02 -> osm_vendor_bind: Binding to port 0x2c903000134f5 Jul 10 13:23:45 628499 [F6AF1030] 0x02 -> osmtest_validate_sa_class_port_info: ----------------------------- SA Class Port Info: base_ver:1 class_ver:2 cap_mask:0x2601 cap_mask2:0x0 resp_time_val:0x14 ----------------------------- Jul 10 13:23:45 631129 [42591940] 0x01 -> __osmv_sa_mad_rcv_cb: ERR 5501: Remote error:0x1C00 Jul 10 13:23:45 631158 [42591940] 0x01 -> osmtest_query_res_cb: ERR 0003: Error on query (IB_REMOTE_ERROR) Jul 10 13:23:45 631189 [F6AF1030] 0x01 -> osmtest_get_all_recs: ERR 0064: ib_query failed (IB_REMOTE_ERROR) Jul 10 13:23:45 631213 [F6AF1030] 0x01 -> osmtest_get_all_recs: Remote error = IB_MAD_STATUS_INVALID_FIELD Jul 10 13:23:45 631245 [F6AF1030] 0x01 -> osmtest_write_all_path_recs: ERR 0025: osmtest_get_all_recs failed (IB_REMOTE_ERROR) Jul 10 13:23:45 631278 [F6AF1030] 0x01 -> osmtest_run: ERR 0139: Inventory file create failed (IB_REMOTE_ERROR) OSMTEST: TEST "Create Inventory" FAIL [root at gaia-08 ~]# echo $? 43 [root at gaia-08 ~]# ifconfig ib0 ib0 Link encap:InfiniBand HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 inet addr:192.168.0.68 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::202:c903:1:34f5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1 RX packets:583403 errors:0 dropped:0 overruns:0 frame:0 TX packets:537773 errors:3 dropped:3 overruns:0 carrier:0 collisions:0 txqueuelen:256 RX bytes:748235387 (713.5 MiB) TX bytes:626888871 (597.8 MiB) Steven Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at mellanox.co.il Thu Jul 10 06:34:39 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Thu, 10 Jul 2008 16:34:39 +0300 Subject: [ofa-general] [PATCH][resend] IB/mlx4: Use kzalloc when allocating a new mlx4_ib_qp Message-ID: <20080710133439.GA20898@mtls03> Current code uses kmalloc() and then does a bitwise or operation on qp->flags at create_qp_common(). This patch uses kzalloc and avoids farther explicit clearing of variables. This is the size of the text segment of mlx4_ib.ko: text before: 24565 after: 24497 change: 68 Signed-off-by: Eli Cohen --- Roland, this patch (or the previous version) is a bug fix and I think it should be pushed to 2.6.27. drivers/infiniband/hw/mlx4/qp.c | 15 ++------------- 1 files changed, 2 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 91590e7..89eb6cb 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -454,19 +454,8 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, spin_lock_init(&qp->rq.lock); qp->state = IB_QPS_RESET; - qp->atomic_rd_en = 0; - qp->resp_depth = 0; - - qp->rq.head = 0; - qp->rq.tail = 0; - qp->sq.head = 0; - qp->sq.tail = 0; - qp->sq_next_wqe = 0; - if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR) qp->sq_signal_bits = cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE); - else - qp->sq_signal_bits = 0; err = set_rq_size(dev, &init_attr->cap, !!pd->uobject, !!init_attr->srq, qp); if (err) @@ -704,7 +693,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, case IB_QPT_UC: case IB_QPT_UD: { - qp = kmalloc(sizeof *qp, GFP_KERNEL); + qp = kzalloc(sizeof *qp, GFP_KERNEL); if (!qp) return ERR_PTR(-ENOMEM); @@ -725,7 +714,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, if (pd->uobject) return ERR_PTR(-EINVAL); - sqp = kmalloc(sizeof *sqp, GFP_KERNEL); + sqp = kzalloc(sizeof *sqp, GFP_KERNEL); if (!sqp) return ERR_PTR(-ENOMEM); -- 1.5.6 From hrosenstock at xsigo.com Thu Jul 10 06:37:52 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 06:37:52 -0700 Subject: [ofa-general] ***SPAM*** osmtest fails to create inventory file In-Reply-To: References: Message-ID: <1215697072.18371.168.camel@hrosenstock-ws.xsigo.com> Hi, On Thu, 2008-07-10 at 19:43 +0800, Wen Hao Wang wrote: > Hello all: > > I am using IBM HS21XM blades with 4X InfiniBand DDR Expansion Card, on > x86_64 RHEL5.2 Server. opensm-3.1.8-1.el5 is installed. The IB device > has worked for weeks. Today I tried "osmtest -f c" to generate > inventory file. But it gave error 5501, 0003 and 0064. Would anyone > give some advices? Thanks! Is OpenSM the master SM or is some other SM master ? From capmask below, it appears some other SM might be in your subnet. What does saquery -s say ? -- Hal > [root at gaia-08 ~]# osmtest -f c > > Command Line Arguments > Done with args > Flow = Create Inventory > Jul 10 13:23:45 588714 [F6AF1030] 0x7f -> Setting log level to: 0x03 > Jul 10 13:23:45 606560 [F6AF1030] 0x02 -> osm_vendor_bind: Binding to > port 0x2c903000134f5 > Jul 10 13:23:45 628499 [F6AF1030] 0x02 -> > osmtest_validate_sa_class_port_info: > ----------------------------- > SA Class Port Info: > base_ver:1 > class_ver:2 > cap_mask:0x2601 > cap_mask2:0x0 > resp_time_val:0x14 > ----------------------------- > Jul 10 13:23:45 631129 [42591940] 0x01 -> __osmv_sa_mad_rcv_cb: ERR > 5501: Remote error:0x1C00 > Jul 10 13:23:45 631158 [42591940] 0x01 -> osmtest_query_res_cb: ERR > 0003: Error on query (IB_REMOTE_ERROR) > Jul 10 13:23:45 631189 [F6AF1030] 0x01 -> osmtest_get_all_recs: ERR > 0064: ib_query failed (IB_REMOTE_ERROR) > Jul 10 13:23:45 631213 [F6AF1030] 0x01 -> osmtest_get_all_recs: Remote > error = IB_MAD_STATUS_INVALID_FIELD > Jul 10 13:23:45 631245 [F6AF1030] 0x01 -> osmtest_write_all_path_recs: > ERR 0025: osmtest_get_all_recs failed (IB_REMOTE_ERROR) > Jul 10 13:23:45 631278 [F6AF1030] 0x01 -> osmtest_run: ERR 0139: > Inventory file create failed (IB_REMOTE_ERROR) > OSMTEST: TEST "Create Inventory" FAIL > [root at gaia-08 ~]# echo $? > 43 > [root at gaia-08 ~]# ifconfig ib0 > ib0 Link encap:InfiniBand HWaddr > 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 > inet addr:192.168.0.68 Bcast:192.168.0.255 Mask:255.255.255.0 > inet6 addr: fe80::202:c903:1:34f5/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1 > RX packets:583403 errors:0 dropped:0 overruns:0 frame:0 > TX packets:537773 errors:3 dropped:3 overruns:0 carrier:0 > collisions:0 txqueuelen:256 > RX bytes:748235387 (713.5 MiB) TX bytes:626888871 (597.8 MiB) > > > Steven Wang > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From bs at q-leap.de Thu Jul 10 06:38:56 2008 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 10 Jul 2008 15:38:56 +0200 Subject: [ofa-general] debian build instructions? In-Reply-To: <4875FB8B.4020507@ec.gc.ca> References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> Message-ID: <200807101538.56915.bs@q-leap.de> On Thursday 10 July 2008 14:07:39 John Marshall wrote: > John Marshall wrote: > > Hi, > > > > I have muddled my way through building the latest > > ofed-1.3.1 on debian etch. Is there a simple way of > > doing this that I am missing? I have searched for info > > and found some, but nothing that gives a step by > > step procedure. > > There seem to be a lot of people on this list. Is there > no one working with debian that might be able to > give a pointer? Actually we are creating debian packages of ofed-1.3. We also always had packages for ofed-1.2, but the huge package split up of ofed-1.3 means quite a lot of work. Here is a sources.list entry for etch (amd64) and packages we already have (sources packages will follow later on). deb http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/infiniband/ ./ Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH From mdidomenico4 at gmail.com Thu Jul 10 06:49:54 2008 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Thu, 10 Jul 2008 09:49:54 -0400 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> Message-ID: tests using connected mode IPoIB dmesg from the compute node cfd-cnsl-0001:~ # dmesg ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:0091 MTU > 0 ib0: PathRec LID 0x0518 for GID fe80:0000:0000:0000:00e0:8111:0100:0091 ib0: Created ah ffff810216a8c740 ib0: created address handle ffff81041c6533c0 for LID 0x0518, SL 0 ib0: Request connection 0x4a for gid fe80:0000:0000:0000:00e0:8111:0100:0091 qpn 0x404 ib0: REP received. ib0: REQ arrived ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:007d MTU > 0 ib0: PathRec LID 0x0161 for GID fe80:0000:0000:0000:00e0:8111:0100:007d ib0: Created ah ffff810216a8c580 ib0: created address handle ffff81041e564d40 for LID 0x0161, SL 0 ib0: Send unicast ARP to 0161 ib0: REQ arrived ib0: Send unicast ARP to 0161 ib0: REQ arrived ib0: Send unicast ARP to 0161 ib0: Send unicast ARP to 0518 ib0: REQ arrived ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0518 ib0: Send unicast ARP to 0161 ib0: REQ arrived ib0: DREQ received. ib0: CM error 9. ib0: Destroy active connection 0x4a head 0x19680 tail 0x19680 dmesg from the io node [root at cfd-io-0001 ~]# dmesg ib0: Start path record lookup for fe80:0000:0000:0000:0002:c903:0000:c36d MTU > 0 ib0: PathRec LID 0x0384 for GID fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Created ah ffff810126e93500 ib0: created address handle ffff81012b98cc80 for LID 0x0384, SL 0 ib0: Send unicast ARP to 0384 ib0: REQ arrived ib0: Request connection 0x10406 for gid fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 ib0: REP received. ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Destroy active connection 0x10406 head 0x12455 tail 0x12455 ib0: Request connection 0x30406 for gid fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 ib0: REP received. ib0: Request connection 0x30407 for gid fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 ib0: REP received. ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Destroy active connection 0x30406 head 0x8 tail 0x8 ib0: Request connection 0x40406 for gid fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 ib0: REP received. ib0: Send unicast ARP to 0384 ib0: Send unicast ARP to 0384 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 ib0: Destroy active connection 0x40406 head 0x8 tail 0x8 ib0: neigh_destructor for bonding device: ib0 ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d ib0: Destroy active connection 0x30407 head 0x3 tail 0x3 ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: Request connection 0x80405 for gid fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 ib0: REQ arrived ib0: REP received. ib0: Send unicast ARP to 045e ib0: REQ arrived ib0: Send unicast ARP to 045e On Thu, Jul 10, 2008 at 9:21 AM, Michael Di Domenico wrote: > tests using datagram IPoIB (non-connected mode) > dmesg from the compute node > cfd-cnsl-0001:~ # dmesg > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:007d > MTU > 1024 > ib0: PathRec LID 0x0161 for GID fe80:0000:0000:0000:00e0:8111:0100:007d > ib0: Created ah ffff81042063dc80 > ib0: created address handle ffff8102206144c0 for LID 0x0161, SL 0 > ib0: Send unicast ARP to 0161 > ib0: Send unicast ARP to 0161 > ib0: Send unicast ARP to 0161 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0518 > ib0: Send unicast ARP to 0161 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000404 fe80:0000:0000:0000:00e0:8111:0100:0091 > > > dmesg from the IO node > > [root at cfd-io-0001 ~]# dmesg > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 0384 > ib_mthca 0000:07:00.0: too many gathers > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 0384 > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 045e > ib0: REQ arrived > ib0: Send unicast ARP to 0384 > ib0: Send unicast ARP to 045e > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 0384 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib0: REQ arrived > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib_mthca 0000:07:00.0: opcode invalid > ib0: post_send failed > ib0: Send unicast ARP to 0384 > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d > ib0: Send unicast ARP to 045e > ib0: neigh_destructor for bonding device: ib0 > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 > > > On Thu, Jul 10, 2008 at 8:33 AM, Eli Cohen wrote: > >> On Thu, Jul 10, 2008 at 07:57:30AM -0400, Michael Di Domenico wrote: >> > maybe i spoke too soon, so more output came, i thought it was done >> > ib0: mtu > 2044 will cause multicast packet drops. >> > eth5: no IPv6 routers present >> > ib0: Send unicast ARP to 0384 >> > ib0: REQ arrived >> > ib0: Request connection 0x2c0406 for gid >> > fe80:0000:0000:0000:0002:c903:0000:c36d qpn 0x48 >> > ib0: REP received. >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 045e >> > ib0: REQ arrived >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 0384 >> > ib0: Send unicast ARP to 045e >> > ib0: REQ arrived >> > ib0: neigh_destructor for bonding device: ib0 >> > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:c36d >> > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:c36d >> > ib0: Destroy active connection 0x2c0406 head 0x22644 tail 0x22644 >> > ib0: Request connection 0x2f0406 for gid >> > fe80:0000:0000:0000:0002:c903:0000:cad1 qpn 0x48 >> > ib0: REP received. >> > ib0: neigh_destructor for bonding device: ib0 >> > ib0: neigh_cleanup for 000048 fe80:0000:0000:0000:0002:c903:0000:cad1 >> > ib0: Reap connection for gid fe80:0000:0000:0000:0002:c903:0000:cad1 >> > ib0: Destroy active connection 0x2f0406 head 0x4 tail 0x4 >> >> >> I see you're working in connected mode. Can you please do the >> follwoing: >> >> 1. clear dmesg: dmesg -c >> 2. run again, then send all the output of dmesg >> >> Do this for both connected and datagram modes. >> >> Thanks. >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronli.voltaire at gmail.com Thu Jul 10 07:35:24 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Thu, 10 Jul 2008 17:35:24 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH 3/3 v4] ib/ipoib: blocking multicast loopback ipoib packets In-Reply-To: <200807101110.56602.jackm@dev.mellanox.co.il> References: <200807101039.41832.jackm@dev.mellanox.co.il> <3b5e77ad0807100047u1790dd7chf5b59f14be0684a4@mail.gmail.com> <200807101110.56602.jackm@dev.mellanox.co.il> Message-ID: <3b5e77ad0807100735y3af99636rbba2fe1eac51191c@mail.gmail.com> OK, thank you. If you'll send them until Sunday, I'll send mine on Sunday Ron On Thu, Jul 10, 2008 at 11:10 AM, Jack Morgenstein wrote: > On Thursday 10 July 2008 10:47, Ron Livne wrote: >> >> I'm going to release some patches, based on your XRC patches, in which >> I'm going to replace xrc_ops with more_ops. >> These patches will will be relative to your patches. >> I hope it's ok with you. >> >> Ron >> > It is OK with me. > > I will make the name change (xrc_ops to more_ops) already in the updated patch set > I'll be submitting later today to the list. > > You can then add your changes on top of mine. > > - Jack > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From eli at dev.mellanox.co.il Thu Jul 10 08:30:46 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 10 Jul 2008 18:30:46 +0300 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> Message-ID: <20080710153046.GB1166@mtls03> On Thu, Jul 10, 2008 at 09:49:54AM -0400, Michael Di Domenico wrote: > > > tests using datagram IPoIB (non-connected mode) > > dmesg from the compute node > > cfd-cnsl-0001:~ # dmesg > > ib0: Send unicast ARP to 0518 > > ib0: Send unicast ARP to 0518 > > ib0: Send unicast ARP to 0518 > > ib0: Send unicast ARP to 0518 > > ib0: Start path record lookup for fe80:0000:0000:0000:00e0:8111:0100:007d > > MTU > 1024 > > ib0: PathRec LID 0x0161 for GID fe80:0000:0000:0000:00e0:8111:0100:007d > > ib0: Created ah ffff81042063dc80 > > ib0: created address handle ffff8102206144c0 for LID 0x0161, SL 0 > > ib0: Send unicast ARP to 0161 > > ib0: Send unicast ARP to 0161 > > ib0: Send unicast ARP to 0161 > > ib0: Send unicast ARP to 0518 > > ib0: Send unicast ARP to 0518 > > ib0: Send unicast ARP to 0518 > > ib0: Send unicast ARP to 0161 > > ib0: neigh_destructor for bonding device: ib0 > > ib0: neigh_cleanup for 000404 fe80:0000:0000:0000:00e0:8111:0100:0091 Here I don't see any problem. > > > > > > dmesg from the IO node > > > > [root at cfd-io-0001 ~]# dmesg > > ib0: Send unicast ARP to 0384 > > ib0: Send unicast ARP to 0384 > > ib_mthca 0000:07:00.0: too many gathers > > ib0: post_send failed > > ib_mthca 0000:07:00.0: opcode invalid > > ib0: post_send failed This looks like a problem that needs investigation but this is not ConnectX. Are you sure you are using only ConnectX? From mdidomenico4 at gmail.com Thu Jul 10 08:39:34 2008 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Thu, 10 Jul 2008 11:39:34 -0400 Subject: ***SPAM*** Re: [ofa-general] IPoIB arps disappearing In-Reply-To: <20080710153046.GB1166@mtls03> References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> <20080710153046.GB1166@mtls03> Message-ID: On Thu, Jul 10, 2008 at 11:30 AM, Eli Cohen wrote: > > > dmesg from the IO node > > > > > > [root at cfd-io-0001 ~]# dmesg > > > ib0: Send unicast ARP to 0384 > > > ib0: Send unicast ARP to 0384 > > > ib_mthca 0000:07:00.0: too many gathers > > > ib0: post_send failed > > > ib_mthca 0000:07:00.0: opcode invalid > > > ib0: post_send failed > This looks like a problem that needs investigation but this is not > ConnectX. Are you sure you are using only ConnectX? > > > Correct. I was mistaken, we're actually using mthca onboard IB adapters for the IO nodes -------------- next part -------------- An HTML attachment was scrubbed... URL: From jackm at dev.mellanox.co.il Thu Jul 10 08:51:10 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:10 +0300 Subject: [ofa-general] [PATCH 0/12 V2] XRC patch series, for 2.6.27 Message-ID: <200807101851.11045.jackm@dev.mellanox.co.il> This is the updated patch series, incorporating all of the fixes and changes based on Roland's and Ron Livne's feedback. This series is ready for Ron's proposed changes to be integrated. This implementation will shortly be put into OFED 1.4. The OFED 1.3 users will need to recompile when OFED 1.4 gets released, due to an ABI alignment bug that Roland spotted (for x86 libraries on x86_64 machines). I've kept the same naming and numbering order for the patches, and added another one at the end for the libibverbs XRC man pages. My apologies, in that I've started using git-format-patch (I used quilt for the previous patch set) so the order of files within patches has changed. - Jack From jackm at dev.mellanox.co.il Thu Jul 10 08:51:33 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:33 +0300 Subject: [ofa-general] [PATCH 4/12 V2] libmlx4: XRC receive-only QP implementation Message-ID: <200807101851.33897.jackm@dev.mellanox.co.il> From e1760ce0a30ff872db3fc066e9cf74b538bb0cb4 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Wed, 9 Jul 2008 13:58:05 +0300 Subject: [PATCH] Added support for XRC receive-only QPs. (OFED 1.3 commit 3869d6dab7e12fe452270ca641f7dd7082b42482) V2: 1. xrc_ops changed to more_ops Signed-off-by: Jack Morgenstein --- src/mlx4.c | 5 +++++ src/mlx4.h | 15 +++++++++++++++ src/verbs.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+), 0 deletions(-) diff --git a/src/mlx4.c b/src/mlx4.c index 8a46543..18333e7 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -73,6 +73,11 @@ static struct ibv_more_ops mlx4_more_ops = { .create_xrc_srq = mlx4_create_xrc_srq, .open_xrc_domain = mlx4_open_xrc_domain, .close_xrc_domain = mlx4_close_xrc_domain, + .create_xrc_rcv_qp = mlx4_create_xrc_rcv_qp, + .modify_xrc_rcv_qp = mlx4_modify_xrc_rcv_qp, + .query_xrc_rcv_qp = mlx4_query_xrc_rcv_qp, + .reg_xrc_rcv_qp = mlx4_reg_xrc_rcv_qp, + .unreg_xrc_rcv_qp = mlx4_unreg_xrc_rcv_qp, #endif }; #endif diff --git a/src/mlx4.h b/src/mlx4.h index 28a8f9c..a791901 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -407,6 +407,21 @@ struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, int fd, int oflag); int mlx4_close_xrc_domain(struct ibv_xrc_domain *d); +int mlx4_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_qp_num); +int mlx4_modify_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, + int attr_mask); +int mlx4_query_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, + int attr_mask, + struct ibv_qp_init_attr *init_attr); +int mlx4_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); +int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); #endif diff --git a/src/verbs.c b/src/verbs.c index b883992..b1bad3c 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -770,4 +770,59 @@ int mlx4_close_xrc_domain(struct ibv_xrc_domain *d) free(d); return 0; } + +int mlx4_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_qp_num) +{ + + return ibv_cmd_create_xrc_rcv_qp(init_attr, xrc_qp_num); +} + +int mlx4_modify_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, + int attr_mask) +{ + return ibv_cmd_modify_xrc_rcv_qp(xrc_domain, xrc_qp_num, + attr, attr_mask); +} + +int mlx4_query_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, + int attr_mask, + struct ibv_qp_init_attr *init_attr) +{ + int ret; + + ret = ibv_cmd_query_xrc_rcv_qp(xrc_domain, xrc_qp_num, + attr, attr_mask, init_attr); + if (ret) + return ret; + + init_attr->cap.max_send_wr = init_attr->cap.max_send_sge = 1; + init_attr->cap.max_recv_sge = init_attr->cap.max_recv_wr = 0; + init_attr->cap.max_inline_data = 0; + init_attr->recv_cq = init_attr->send_cq = NULL; + init_attr->srq = NULL; + init_attr->xrc_domain = xrc_domain; + init_attr->qp_type = IBV_QPT_XRC; + init_attr->qp_context = NULL; + attr->cap = init_attr->cap; + + return 0; +} + +int mlx4_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num) +{ + return ibv_cmd_reg_xrc_rcv_qp(xrc_domain, xrc_qp_num); +} + +int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num) +{ + return ibv_cmd_unreg_xrc_rcv_qp(xrc_domain, xrc_qp_num); +} + #endif -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:37 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:37 +0300 Subject: [ofa-general] [PATCH 5/12 V2] core: XRC base implementation Message-ID: <200807101851.37908.jackm@dev.mellanox.co.il> From 2b577ecf8ef5e8e2e73e8bd743a5e3b746aed642 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Mon, 7 Jul 2008 18:20:49 +0300 Subject: [PATCH] IB/core: Implement XRC support at verbs layer (for case in which fd is not used when opening an xrc_domain). Rev 2: a. is_srq flag indicates ONLY srq. b. Check that for XRC QP's, do not have an SRQ and do have an XRC domain. Rev 3: checkpatch cleanups. Signed-off-by: Jack Morgenstein --- drivers/infiniband/core/uverbs.h | 5 + drivers/infiniband/core/uverbs_cmd.c | 279 ++++++++++++++++++++++++++++++++- drivers/infiniband/core/uverbs_main.c | 35 +++-- drivers/infiniband/core/verbs.c | 77 +++++++++- include/rdma/ib_user_verbs.h | 37 +++++- include/rdma/ib_verbs.h | 35 ++++ 6 files changed, 453 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index b3ea958..6b583c6 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -141,6 +141,7 @@ extern struct idr ib_uverbs_ah_idr; extern struct idr ib_uverbs_cq_idr; extern struct idr ib_uverbs_qp_idr; extern struct idr ib_uverbs_srq_idr; +extern struct idr ib_uverbs_xrc_domain_idr; void idr_remove_uobj(struct idr *idp, struct ib_uobject *uobj); @@ -194,5 +195,9 @@ IB_UVERBS_DECLARE_CMD(create_srq); IB_UVERBS_DECLARE_CMD(modify_srq); IB_UVERBS_DECLARE_CMD(query_srq); IB_UVERBS_DECLARE_CMD(destroy_srq); +IB_UVERBS_DECLARE_CMD(create_xrc_srq); +IB_UVERBS_DECLARE_CMD(open_xrc_domain); +IB_UVERBS_DECLARE_CMD(close_xrc_domain); + #endif /* UVERBS_H */ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 56feab6..bdd1265 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -254,6 +254,17 @@ static void put_srq_read(struct ib_srq *srq) put_uobj_read(srq->uobject); } +static struct ib_xrcd *idr_read_xrcd(int xrcd_handle, + struct ib_ucontext *context) +{ + return idr_read_obj(&ib_uverbs_xrc_domain_idr, xrcd_handle, context, 0); +} + +static void put_xrcd_read(struct ib_xrcd *xrcd) +{ + put_uobj_read(xrcd->uobject); +} + ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) @@ -297,6 +308,7 @@ ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, INIT_LIST_HEAD(&ucontext->qp_list); INIT_LIST_HEAD(&ucontext->srq_list); INIT_LIST_HEAD(&ucontext->ah_list); + INIT_LIST_HEAD(&ucontext->xrc_domain_list); ucontext->closing = 0; resp.num_comp_vectors = file->device->num_comp_vectors; @@ -1026,6 +1038,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, struct ib_srq *srq; struct ib_qp *qp; struct ib_qp_init_attr attr; + struct ib_xrcd *xrcd; int ret; if (out_len < sizeof resp) @@ -1045,13 +1058,17 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, init_uobj(&obj->uevent.uobject, cmd.user_handle, file->ucontext, &qp_lock_key); down_write(&obj->uevent.uobject.mutex); - srq = cmd.is_srq ? idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; + srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? + idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; + xrcd = cmd.qp_type == IB_QPT_XRC ? + idr_read_xrcd(cmd.srq_handle, file->ucontext) : NULL; pd = idr_read_pd(cmd.pd_handle, file->ucontext); scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? scq : idr_read_cq(cmd.recv_cq_handle, file->ucontext, 1); - if (!pd || !scq || !rcq || (cmd.is_srq && !srq)) { + if (!pd || !scq || !rcq || (cmd.is_srq && !srq) || + (cmd.qp_type == IB_QPT_XRC && !xrcd)) { ret = -EINVAL; goto err_put; } @@ -1063,6 +1080,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.srq = srq; attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; + attr.xrc_domain = xrcd; attr.create_flags = 0; attr.cap.max_send_wr = cmd.max_send_wr; @@ -1090,11 +1108,14 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, qp->event_handler = attr.event_handler; qp->qp_context = attr.qp_context; qp->qp_type = attr.qp_type; + qp->xrcd = attr.xrc_domain; atomic_inc(&pd->usecnt); atomic_inc(&attr.send_cq->usecnt); atomic_inc(&attr.recv_cq->usecnt); if (attr.srq) atomic_inc(&attr.srq->usecnt); + else if (attr.xrc_domain) + atomic_inc(&attr.xrc_domain->usecnt); obj->uevent.uobject.object = qp; ret = idr_add_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); @@ -1122,6 +1143,8 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, put_cq_read(rcq); if (srq) put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd); mutex_lock(&file->mutex); list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); @@ -1148,6 +1171,8 @@ err_put: put_cq_read(rcq); if (srq) put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd); put_uobj_write(&obj->uevent.uobject); return ret; @@ -2000,6 +2025,8 @@ ssize_t ib_uverbs_create_srq(struct ib_uverbs_file *file, srq->uobject = &obj->uobject; srq->event_handler = attr.event_handler; srq->srq_context = attr.srq_context; + srq->xrc_cq = NULL; + srq->xrcd = NULL; atomic_inc(&pd->usecnt); atomic_set(&srq->usecnt, 0); @@ -2045,6 +2072,136 @@ err: return ret; } +ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_create_xrc_srq cmd; + struct ib_uverbs_create_srq_resp resp; + struct ib_udata udata; + struct ib_uevent_object *obj; + struct ib_pd *pd; + struct ib_srq *srq; + struct ib_cq *xrc_cq; + struct ib_xrcd *xrcd; + struct ib_srq_init_attr attr; + int ret; + + if (out_len < sizeof resp) + return -ENOSPC; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + INIT_UDATA(&udata, buf + sizeof cmd, + (unsigned long) cmd.response + sizeof resp, + in_len - sizeof cmd, out_len - sizeof resp); + + obj = kmalloc(sizeof *obj, GFP_KERNEL); + if (!obj) + return -ENOMEM; + + init_uobj(&obj->uobject, cmd.user_handle, file->ucontext, + &srq_lock_key); + down_write(&obj->uobject.mutex); + + pd = idr_read_pd(cmd.pd_handle, file->ucontext); + if (!pd) { + ret = -EINVAL; + goto err; + } + + xrc_cq = idr_read_cq(cmd.xrc_cq, file->ucontext, 0); + if (!xrc_cq) { + ret = -EINVAL; + goto err_put_pd; + } + + xrcd = idr_read_xrcd(cmd.xrcd_handle, file->ucontext); + if (!xrcd) { + ret = -EINVAL; + goto err_put_cq; + } + + + attr.event_handler = ib_uverbs_srq_event_handler; + attr.srq_context = file; + attr.attr.max_wr = cmd.max_wr; + attr.attr.max_sge = cmd.max_sge; + attr.attr.srq_limit = cmd.srq_limit; + + obj->events_reported = 0; + INIT_LIST_HEAD(&obj->event_list); + + srq = pd->device->create_xrc_srq(pd, xrc_cq, xrcd, &attr, &udata); + if (IS_ERR(srq)) { + ret = PTR_ERR(srq); + goto err_put; + } + + srq->device = pd->device; + srq->pd = pd; + srq->uobject = &obj->uobject; + srq->event_handler = attr.event_handler; + srq->srq_context = attr.srq_context; + srq->xrc_cq = xrc_cq; + srq->xrcd = xrcd; + atomic_inc(&pd->usecnt); + atomic_inc(&xrc_cq->usecnt); + atomic_inc(&xrcd->usecnt); + + atomic_set(&srq->usecnt, 0); + + obj->uobject.object = srq; + ret = idr_add_uobj(&ib_uverbs_srq_idr, &obj->uobject); + if (ret) + goto err_destroy; + + memset(&resp, 0, sizeof resp); + resp.srq_handle = obj->uobject.id; + resp.max_wr = attr.attr.max_wr; + resp.max_sge = attr.attr.max_sge; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + ret = -EFAULT; + goto err_copy; + } + + put_xrcd_read(xrcd); + put_cq_read(xrc_cq); + put_pd_read(pd); + + mutex_lock(&file->mutex); + list_add_tail(&obj->uobject.list, &file->ucontext->srq_list); + mutex_unlock(&file->mutex); + + obj->uobject.live = 1; + + up_write(&obj->uobject.mutex); + + return in_len; + +err_copy: + idr_remove_uobj(&ib_uverbs_srq_idr, &obj->uobject); + +err_destroy: + ib_destroy_srq(srq); + +err_put: + put_xrcd_read(xrcd); + +err_put_cq: + put_cq_read(xrc_cq); + +err_put_pd: + put_pd_read(pd); + +err: + put_uobj_write(&obj->uobject); + return ret; +} + ssize_t ib_uverbs_modify_srq(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) @@ -2163,3 +2320,121 @@ ssize_t ib_uverbs_destroy_srq(struct ib_uverbs_file *file, return ret ? ret : in_len; } + +ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_open_xrc_domain cmd; + struct ib_uverbs_open_xrc_domain_resp resp; + struct ib_udata udata; + struct ib_uobject *uobj; + struct ib_xrcd *xrcd; + int ret; + + if (out_len < sizeof resp) + return -ENOSPC; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + /* file descriptors/inodes not yet implemented */ + if (cmd.fd != (u32) (-1)) + return -ENOSYS; + + INIT_UDATA(&udata, buf + sizeof cmd, + (unsigned long) cmd.response + sizeof resp, + in_len - sizeof cmd, out_len - sizeof resp); + + uobj = kmalloc(sizeof *uobj, GFP_KERNEL); + if (!uobj) + return -ENOMEM; + + init_uobj(uobj, 0, file->ucontext, &pd_lock_key); + down_write(&uobj->mutex); + + + xrcd = file->device->ib_dev->alloc_xrcd(file->device->ib_dev, + file->ucontext, &udata); + if (IS_ERR(xrcd)) { + ret = PTR_ERR(xrcd); + goto err; + } + + xrcd->fd = cmd.fd; + xrcd->flags = cmd.oflags; + xrcd->uobject = uobj; + xrcd->device = file->device->ib_dev; + atomic_set(&xrcd->usecnt, 0); + + uobj->object = xrcd; + ret = idr_add_uobj(&ib_uverbs_xrc_domain_idr, uobj); + if (ret) + goto err_idr; + + memset(&resp, 0, sizeof resp); + resp.xrcd_handle = uobj->id; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + ret = -EFAULT; + goto err_copy; + } + + mutex_lock(&file->mutex); + list_add_tail(&uobj->list, &file->ucontext->xrc_domain_list); + mutex_unlock(&file->mutex); + + uobj->live = 1; + + up_write(&uobj->mutex); + + return in_len; + +err_copy: + idr_remove_uobj(&ib_uverbs_pd_idr, uobj); + +err_idr: + ib_dealloc_xrcd(xrcd); + +err: + put_uobj_write(uobj); + return ret; +} + +ssize_t ib_uverbs_close_xrc_domain(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_close_xrc_domain cmd; + struct ib_uobject *uobj; + int ret; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + uobj = idr_write_uobj(&ib_uverbs_xrc_domain_idr, cmd.xrcd_handle, + file->ucontext); + if (!uobj) + return -EINVAL; + + ret = ib_dealloc_xrcd(uobj->object); + if (!ret) + uobj->live = 0; + + put_uobj_write(uobj); + + if (ret) + return ret; + + idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); + + mutex_lock(&file->mutex); + list_del(&uobj->list); + mutex_unlock(&file->mutex); + + put_uobj(uobj); + + return in_len; +} + diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 965c4be..c3d1ffb 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -72,6 +72,7 @@ DEFINE_IDR(ib_uverbs_ah_idr); DEFINE_IDR(ib_uverbs_cq_idr); DEFINE_IDR(ib_uverbs_qp_idr); DEFINE_IDR(ib_uverbs_srq_idr); +DEFINE_IDR(ib_uverbs_xrc_domain_idr); static spinlock_t map_lock; static struct ib_uverbs_device *dev_table[IB_UVERBS_MAX_DEVICES]; @@ -108,6 +109,9 @@ static ssize_t (*uverbs_cmd_table[])(struct ib_uverbs_file *file, [IB_USER_VERBS_CMD_MODIFY_SRQ] = ib_uverbs_modify_srq, [IB_USER_VERBS_CMD_QUERY_SRQ] = ib_uverbs_query_srq, [IB_USER_VERBS_CMD_DESTROY_SRQ] = ib_uverbs_destroy_srq, + [IB_USER_VERBS_CMD_CREATE_XRC_SRQ] = ib_uverbs_create_xrc_srq, + [IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN] = ib_uverbs_open_xrc_domain, + [IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN] = ib_uverbs_close_xrc_domain, }; static struct vfsmount *uverbs_event_mnt; @@ -211,17 +215,6 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uqp); } - list_for_each_entry_safe(uobj, tmp, &context->cq_list, list) { - struct ib_cq *cq = uobj->object; - struct ib_uverbs_event_file *ev_file = cq->cq_context; - struct ib_ucq_object *ucq = - container_of(uobj, struct ib_ucq_object, uobject); - - idr_remove_uobj(&ib_uverbs_cq_idr, uobj); - ib_destroy_cq(cq); - ib_uverbs_release_ucq(file, ev_file, ucq); - kfree(ucq); - } list_for_each_entry_safe(uobj, tmp, &context->srq_list, list) { struct ib_srq *srq = uobj->object; @@ -234,6 +227,18 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uevent); } + list_for_each_entry_safe(uobj, tmp, &context->cq_list, list) { + struct ib_cq *cq = uobj->object; + struct ib_uverbs_event_file *ev_file = cq->cq_context; + struct ib_ucq_object *ucq = + container_of(uobj, struct ib_ucq_object, uobject); + + idr_remove_uobj(&ib_uverbs_cq_idr, uobj); + ib_destroy_cq(cq); + ib_uverbs_release_ucq(file, ev_file, ucq); + kfree(ucq); + } + /* XXX Free MWs */ list_for_each_entry_safe(uobj, tmp, &context->mr_list, list) { @@ -244,6 +249,14 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uobj); } + list_for_each_entry_safe(uobj, tmp, &context->xrc_domain_list, list) { + struct ib_xrcd *xrcd = uobj->object; + + idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); + ib_dealloc_xrcd(xrcd); + kfree(uobj); + } + list_for_each_entry_safe(uobj, tmp, &context->pd_list, list) { struct ib_pd *pd = uobj->object; diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index a7da9be..1b60806 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -234,6 +234,8 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd, srq->uobject = NULL; srq->event_handler = srq_init_attr->event_handler; srq->srq_context = srq_init_attr->srq_context; + srq->xrc_cq = NULL; + srq->xrcd = NULL; atomic_inc(&pd->usecnt); atomic_set(&srq->usecnt, 0); } @@ -263,16 +265,25 @@ EXPORT_SYMBOL(ib_query_srq); int ib_destroy_srq(struct ib_srq *srq) { struct ib_pd *pd; + struct ib_cq *xrc_cq; + struct ib_xrcd *xrcd; int ret; if (atomic_read(&srq->usecnt)) return -EBUSY; pd = srq->pd; + xrc_cq = srq->xrc_cq; + xrcd = srq->xrcd; ret = srq->device->destroy_srq(srq); - if (!ret) + if (!ret) { atomic_dec(&pd->usecnt); + if (xrc_cq) + atomic_dec(&xrc_cq->usecnt); + if (xrcd) + atomic_dec(&xrcd->usecnt); + } return ret; } @@ -297,6 +308,7 @@ struct ib_qp *ib_create_qp(struct ib_pd *pd, qp->event_handler = qp_init_attr->event_handler; qp->qp_context = qp_init_attr->qp_context; qp->qp_type = qp_init_attr->qp_type; + qp->xrcd = NULL; atomic_inc(&pd->usecnt); atomic_inc(&qp_init_attr->send_cq->usecnt); atomic_inc(&qp_init_attr->recv_cq->usecnt); @@ -327,6 +339,9 @@ static const struct { [IB_QPT_RC] = (IB_QP_PKEY_INDEX | IB_QP_PORT | IB_QP_ACCESS_FLAGS), + [IB_QPT_XRC] = (IB_QP_PKEY_INDEX | + IB_QP_PORT | + IB_QP_ACCESS_FLAGS), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -349,6 +364,9 @@ static const struct { [IB_QPT_RC] = (IB_QP_PKEY_INDEX | IB_QP_PORT | IB_QP_ACCESS_FLAGS), + [IB_QPT_XRC] = (IB_QP_PKEY_INDEX | + IB_QP_PORT | + IB_QP_ACCESS_FLAGS), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -368,6 +386,12 @@ static const struct { IB_QP_RQ_PSN | IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER), + [IB_QPT_XRC] = (IB_QP_AV | + IB_QP_PATH_MTU | + IB_QP_DEST_QPN | + IB_QP_RQ_PSN | + IB_QP_MAX_DEST_RD_ATOMIC | + IB_QP_MIN_RNR_TIMER), }, .opt_param = { [IB_QPT_UD] = (IB_QP_PKEY_INDEX | @@ -378,6 +402,9 @@ static const struct { [IB_QPT_RC] = (IB_QP_ALT_PATH | IB_QP_ACCESS_FLAGS | IB_QP_PKEY_INDEX), + [IB_QPT_XRC] = (IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_PKEY_INDEX), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -398,6 +425,11 @@ static const struct { IB_QP_RNR_RETRY | IB_QP_SQ_PSN | IB_QP_MAX_QP_RD_ATOMIC), + [IB_QPT_XRC] = (IB_QP_TIMEOUT | + IB_QP_RETRY_CNT | + IB_QP_RNR_RETRY | + IB_QP_SQ_PSN | + IB_QP_MAX_QP_RD_ATOMIC), [IB_QPT_SMI] = IB_QP_SQ_PSN, [IB_QPT_GSI] = IB_QP_SQ_PSN, }, @@ -413,6 +445,11 @@ static const struct { IB_QP_ACCESS_FLAGS | IB_QP_MIN_RNR_TIMER | IB_QP_PATH_MIG_STATE), + [IB_QPT_XRC] = (IB_QP_CUR_STATE | + IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_MIN_RNR_TIMER | + IB_QP_PATH_MIG_STATE), [IB_QPT_SMI] = (IB_QP_CUR_STATE | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_CUR_STATE | @@ -437,6 +474,11 @@ static const struct { IB_QP_ALT_PATH | IB_QP_PATH_MIG_STATE | IB_QP_MIN_RNR_TIMER), + [IB_QPT_XRC] = (IB_QP_CUR_STATE | + IB_QP_ACCESS_FLAGS | + IB_QP_ALT_PATH | + IB_QP_PATH_MIG_STATE | + IB_QP_MIN_RNR_TIMER), [IB_QPT_SMI] = (IB_QP_CUR_STATE | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_CUR_STATE | @@ -449,6 +491,7 @@ static const struct { [IB_QPT_UD] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_UC] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_RC] = IB_QP_EN_SQD_ASYNC_NOTIFY, + [IB_QPT_XRC] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_SMI] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_GSI] = IB_QP_EN_SQD_ASYNC_NOTIFY } @@ -471,6 +514,11 @@ static const struct { IB_QP_ACCESS_FLAGS | IB_QP_MIN_RNR_TIMER | IB_QP_PATH_MIG_STATE), + [IB_QPT_XRC] = (IB_QP_CUR_STATE | + IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_MIN_RNR_TIMER | + IB_QP_PATH_MIG_STATE), [IB_QPT_SMI] = (IB_QP_CUR_STATE | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_CUR_STATE | @@ -499,6 +547,18 @@ static const struct { IB_QP_PKEY_INDEX | IB_QP_MIN_RNR_TIMER | IB_QP_PATH_MIG_STATE), + [IB_QPT_XRC] = (IB_QP_PORT | + IB_QP_AV | + IB_QP_TIMEOUT | + IB_QP_RETRY_CNT | + IB_QP_RNR_RETRY | + IB_QP_MAX_QP_RD_ATOMIC | + IB_QP_MAX_DEST_RD_ATOMIC | + IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_PKEY_INDEX | + IB_QP_MIN_RNR_TIMER | + IB_QP_PATH_MIG_STATE), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -583,12 +643,14 @@ int ib_destroy_qp(struct ib_qp *qp) struct ib_pd *pd; struct ib_cq *scq, *rcq; struct ib_srq *srq; + struct ib_xrcd *xrcd; int ret; pd = qp->pd; scq = qp->send_cq; rcq = qp->recv_cq; srq = qp->srq; + xrcd = qp->xrcd; ret = qp->device->destroy_qp(qp); if (!ret) { @@ -597,6 +659,8 @@ int ib_destroy_qp(struct ib_qp *qp) atomic_dec(&rcq->usecnt); if (srq) atomic_dec(&srq->usecnt); + if (xrcd) + atomic_dec(&xrcd->usecnt); } return ret; @@ -904,3 +968,14 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid) return qp->device->detach_mcast(qp, gid, lid); } EXPORT_SYMBOL(ib_detach_mcast); + +int ib_dealloc_xrcd(struct ib_xrcd *xrcd) +{ + if (atomic_read(&xrcd->usecnt)) + return -EBUSY; + + return xrcd->device->dealloc_xrcd(xrcd); +} +EXPORT_SYMBOL(ib_dealloc_xrcd); + + diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h index a17f771..87ea38d 100644 --- a/include/rdma/ib_user_verbs.h +++ b/include/rdma/ib_user_verbs.h @@ -81,7 +81,10 @@ enum { IB_USER_VERBS_CMD_MODIFY_SRQ, IB_USER_VERBS_CMD_QUERY_SRQ, IB_USER_VERBS_CMD_DESTROY_SRQ, - IB_USER_VERBS_CMD_POST_SRQ_RECV + IB_USER_VERBS_CMD_POST_SRQ_RECV, + IB_USER_VERBS_CMD_CREATE_XRC_SRQ, + IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN, + IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN }; /* @@ -647,6 +650,18 @@ struct ib_uverbs_create_srq { __u64 driver_data[0]; }; +struct ib_uverbs_create_xrc_srq { + __u64 response; + __u64 user_handle; + __u32 pd_handle; + __u32 max_wr; + __u32 max_sge; + __u32 srq_limit; + __u32 xrcd_handle; + __u32 xrc_cq; + __u64 driver_data[0]; +}; + struct ib_uverbs_create_srq_resp { __u32 srq_handle; __u32 max_wr; @@ -686,4 +701,24 @@ struct ib_uverbs_destroy_srq_resp { __u32 events_reported; }; +struct ib_uverbs_open_xrc_domain { + __u64 response; + __u32 fd; + __u32 oflags; + __u64 driver_data[0]; +}; + +struct ib_uverbs_open_xrc_domain_resp { + __u32 xrcd_handle; +}; + +struct ib_uverbs_close_xrc_domain { + __u64 response; + __u32 xrcd_handle; + __u32 reserved; + __u64 driver_data[0]; +}; + + + #endif /* IB_USER_VERBS_H */ diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 07b41e0..85080ac 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -103,6 +103,7 @@ enum ib_device_cap_flags { */ IB_DEVICE_UD_IP_CSUM = (1<<18), IB_DEVICE_UD_TSO = (1<<19), + IB_DEVICE_XRC = (1<<20), IB_DEVICE_MEM_MGT_EXTENSIONS = (1<<21), IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22), }; @@ -551,6 +552,7 @@ enum ib_qp_type { IB_QPT_RC, IB_QPT_UC, IB_QPT_UD, + IB_QPT_XRC, IB_QPT_RAW_IPV6, IB_QPT_RAW_ETY }; @@ -570,6 +572,7 @@ struct ib_qp_init_attr { enum ib_sig_type sq_sig_type; enum ib_qp_type qp_type; enum ib_qp_create_flags create_flags; + struct ib_xrcd *xrc_domain; /* XRC qp's only */ u8 port_num; /* special QP types only */ }; @@ -814,6 +817,7 @@ struct ib_ucontext { struct list_head qp_list; struct list_head srq_list; struct list_head ah_list; + struct list_head xrc_domain_list; int closing; }; @@ -841,6 +845,18 @@ struct ib_pd { atomic_t usecnt; /* count all resources */ }; +struct ib_xrcd { + struct ib_device *device; + struct ib_uobject *uobject; + struct rb_node node; + u32 xrc_domain_num; + struct inode *inode; + int fd; + u32 flags; + atomic_t usecnt; /* count all resources */ +}; + + struct ib_ah { struct ib_device *device; struct ib_pd *pd; @@ -862,6 +878,8 @@ struct ib_cq { struct ib_srq { struct ib_device *device; struct ib_pd *pd; + struct ib_cq *xrc_cq; + struct ib_xrcd *xrcd; struct ib_uobject *uobject; void (*event_handler)(struct ib_event *, void *); void *srq_context; @@ -879,6 +897,7 @@ struct ib_qp { void *qp_context; u32 qp_num; enum ib_qp_type qp_type; + struct ib_xrcd *xrcd; /* XRC QPs only */ }; struct ib_mr { @@ -1130,6 +1149,15 @@ struct ib_device { struct ib_grh *in_grh, struct ib_mad *in_mad, struct ib_mad *out_mad); + struct ib_srq * (*create_xrc_srq)(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *srq_init_attr, + struct ib_udata *udata); + struct ib_xrcd * (*alloc_xrcd)(struct ib_device *device, + struct ib_ucontext *context, + struct ib_udata *udata); + int (*dealloc_xrcd)(struct ib_xrcd *xrcd); struct ib_dma_mapping_ops *dma_ops; @@ -2030,4 +2058,11 @@ int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid); */ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid); + +/** + * ib_dealloc_xrcd - Deallocates an extended reliably connected domain. + * @pd: The xrc domain to deallocate. + */ +int ib_dealloc_xrcd(struct ib_xrcd *xrcd); + #endif /* IB_VERBS_H */ -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:42 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:42 +0300 Subject: [ofa-general] [PATCH 6/12 V2] core: XRC - file descriptors associated with XRC domains Message-ID: <200807101851.42625.jackm@dev.mellanox.co.il> From fc8eabcb05965539d485785c94fc3555112a979a Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Mon, 7 Jul 2008 18:33:00 +0300 Subject: [PATCH] ib/core: Add XRC support for userspace file descriptors. Add XRC support for working with file descriptors, to allow sharing XRC domains between processes. v2: checkpatch cleanups Signed-off-by: Jack Morgenstein --- drivers/infiniband/core/device.c | 2 + drivers/infiniband/core/uverbs.h | 2 + drivers/infiniband/core/uverbs_cmd.c | 262 ++++++++++++++++++++++++++++----- drivers/infiniband/core/uverbs_main.c | 4 +- include/rdma/ib_verbs.h | 9 +- 5 files changed, 238 insertions(+), 41 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 7913b80..fdf7875 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -288,6 +288,8 @@ int ib_register_device(struct ib_device *device) INIT_LIST_HEAD(&device->client_data_list); spin_lock_init(&device->event_handler_lock); spin_lock_init(&device->client_data_lock); + device->ib_uverbs_xrcd_table = RB_ROOT; + mutex_init(&device->xrcd_table_mutex); ret = read_port_table_lengths(device); if (ret) { diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index 6b583c6..df9b6d2 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -161,6 +161,8 @@ void ib_uverbs_qp_event_handler(struct ib_event *event, void *context_ptr); void ib_uverbs_srq_event_handler(struct ib_event *event, void *context_ptr); void ib_uverbs_event_handler(struct ib_event_handler *handler, struct ib_event *event); +void ib_uverbs_dealloc_xrcd(struct ib_device *ib_dev, + struct ib_xrcd *xrcd); #define IB_UVERBS_DECLARE_CMD(name) \ ssize_t ib_uverbs_##name(struct ib_uverbs_file *file, \ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index bdd1265..6b333ca 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -37,6 +37,7 @@ #include #include +#include #include "uverbs.h" @@ -255,14 +256,17 @@ static void put_srq_read(struct ib_srq *srq) } static struct ib_xrcd *idr_read_xrcd(int xrcd_handle, - struct ib_ucontext *context) + struct ib_ucontext *context, + struct ib_uobject **uobj) { - return idr_read_obj(&ib_uverbs_xrc_domain_idr, xrcd_handle, context, 0); + *uobj = idr_read_uobj(&ib_uverbs_xrc_domain_idr, xrcd_handle, + context, 0); + return *uobj ? (*uobj)->object : NULL; } -static void put_xrcd_read(struct ib_xrcd *xrcd) +static void put_xrcd_read(struct ib_uobject *uobj) { - put_uobj_read(xrcd->uobject); + put_uobj_read(uobj); } ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, @@ -1039,6 +1043,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, struct ib_qp *qp; struct ib_qp_init_attr attr; struct ib_xrcd *xrcd; + struct ib_uobject *xrcd_uobj; int ret; if (out_len < sizeof resp) @@ -1061,7 +1066,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; xrcd = cmd.qp_type == IB_QPT_XRC ? - idr_read_xrcd(cmd.srq_handle, file->ucontext) : NULL; + idr_read_xrcd(cmd.srq_handle, file->ucontext, &xrcd_uobj) : NULL; pd = idr_read_pd(cmd.pd_handle, file->ucontext); scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? @@ -1144,7 +1149,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, if (srq) put_srq_read(srq); if (xrcd) - put_xrcd_read(xrcd); + put_xrcd_read(xrcd_uobj); mutex_lock(&file->mutex); list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); @@ -1172,7 +1177,7 @@ err_put: if (srq) put_srq_read(srq); if (xrcd) - put_xrcd_read(xrcd); + put_xrcd_read(xrcd_uobj); put_uobj_write(&obj->uevent.uobject); return ret; @@ -2085,6 +2090,7 @@ ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, struct ib_cq *xrc_cq; struct ib_xrcd *xrcd; struct ib_srq_init_attr attr; + struct ib_uobject *xrcd_uobj; int ret; if (out_len < sizeof resp) @@ -2117,7 +2123,7 @@ ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, goto err_put_pd; } - xrcd = idr_read_xrcd(cmd.xrcd_handle, file->ucontext); + xrcd = idr_read_xrcd(cmd.xrcd_handle, file->ucontext, &xrcd_uobj); if (!xrcd) { ret = -EINVAL; goto err_put_cq; @@ -2168,7 +2174,7 @@ ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, goto err_copy; } - put_xrcd_read(xrcd); + put_xrcd_read(xrcd_uobj); put_cq_read(xrc_cq); put_pd_read(pd); @@ -2189,7 +2195,7 @@ err_destroy: ib_destroy_srq(srq); err_put: - put_xrcd_read(xrcd); + put_xrcd_read(xrcd_uobj); err_put_cq: put_cq_read(xrc_cq); @@ -2321,6 +2327,99 @@ ssize_t ib_uverbs_destroy_srq(struct ib_uverbs_file *file, return ret ? ret : in_len; } +static struct inode *xrc_file2inode(struct file *f) +{ + return f->f_dentry->d_inode; +} + +struct xrcd_table_entry { + struct rb_node node; + struct inode *inode; + struct ib_xrcd *xrcd; +}; + +static int xrcd_table_insert(struct ib_device *dev, + struct inode *i_n, + struct ib_xrcd *xrcd) +{ + struct xrcd_table_entry *entry, *scan; + struct rb_node **p = &dev->ib_uverbs_xrcd_table.rb_node; + struct rb_node *parent = NULL; + + entry = kmalloc(sizeof(struct xrcd_table_entry), GFP_KERNEL); + if (!entry) + return -ENOMEM; + + entry->inode = i_n; + entry->xrcd = xrcd; + + while (*p) { + parent = *p; + scan = rb_entry(parent, struct xrcd_table_entry, node); + + if (i_n < scan->inode) + p = &(*p)->rb_left; + else if (i_n > scan->inode) + p = &(*p)->rb_right; + else { + kfree(entry); + return -EEXIST; + } + } + + rb_link_node(&entry->node, parent, p); + rb_insert_color(&entry->node, &dev->ib_uverbs_xrcd_table); + igrab(i_n); + return 0; +} + +static struct xrcd_table_entry *xrcd_table_search(struct ib_device *dev, + struct inode *i_n) +{ + struct xrcd_table_entry *scan; + struct rb_node **p = &dev->ib_uverbs_xrcd_table.rb_node; + struct rb_node *parent = NULL; + + while (*p) { + parent = *p; + scan = rb_entry(parent, struct xrcd_table_entry, node); + + if (i_n < scan->inode) + p = &(*p)->rb_left; + else if (i_n > scan->inode) + p = &(*p)->rb_right; + else + return scan; + } + return NULL; +} + +static int find_xrcd(struct ib_device *dev, struct inode *i_n, + struct ib_xrcd **xrcd) +{ + struct xrcd_table_entry *entry; + + entry = xrcd_table_search(dev, i_n); + if (!entry) + return -EINVAL; + + *xrcd = entry->xrcd; + return 0; +} + + +static void xrcd_table_delete(struct ib_device *dev, + struct inode *i_n) +{ + struct xrcd_table_entry *entry = xrcd_table_search(dev, i_n); + + if (entry) { + iput(i_n); + rb_erase(&entry->node, &dev->ib_uverbs_xrcd_table); + kfree(entry); + } +} + ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) @@ -2329,8 +2428,11 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, struct ib_uverbs_open_xrc_domain_resp resp; struct ib_udata udata; struct ib_uobject *uobj; - struct ib_xrcd *xrcd; - int ret; + struct ib_xrcd *xrcd = NULL; + struct file *f = NULL; + struct inode *inode = NULL; + int ret = 0; + int new_xrcd = 0; if (out_len < sizeof resp) return -ENOSPC; @@ -2338,35 +2440,61 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, if (copy_from_user(&cmd, buf, sizeof cmd)) return -EFAULT; - /* file descriptors/inodes not yet implemented */ - if (cmd.fd != (u32) (-1)) - return -ENOSYS; - INIT_UDATA(&udata, buf + sizeof cmd, (unsigned long) cmd.response + sizeof resp, in_len - sizeof cmd, out_len - sizeof resp); + mutex_lock(&file->device->ib_dev->xrcd_table_mutex); + if (cmd.fd != (u32) (-1)) { + /* search for file descriptor */ + f = fget(cmd.fd); + if (!f) { + ret = -EBADF; + goto err_table_mutex_unlock; + } + + inode = xrc_file2inode(f); + if (!inode) { + ret = -EBADF; + goto err_table_mutex_unlock; + } + + ret = find_xrcd(file->device->ib_dev, inode, &xrcd); + if (ret && !(cmd.oflags & O_CREAT)) { + /* no file descriptor. Need CREATE flag */ + ret = -EAGAIN; + goto err_table_mutex_unlock; + } + + if (xrcd && cmd.oflags & O_EXCL) { + ret = -EINVAL; + goto err_table_mutex_unlock; + } + } + uobj = kmalloc(sizeof *uobj, GFP_KERNEL); - if (!uobj) - return -ENOMEM; + if (!uobj) { + ret = -ENOMEM; + goto err_table_mutex_unlock; + } init_uobj(uobj, 0, file->ucontext, &pd_lock_key); down_write(&uobj->mutex); - - xrcd = file->device->ib_dev->alloc_xrcd(file->device->ib_dev, - file->ucontext, &udata); - if (IS_ERR(xrcd)) { - ret = PTR_ERR(xrcd); - goto err; + if (!xrcd) { + xrcd = file->device->ib_dev->alloc_xrcd(file->device->ib_dev, + file->ucontext, &udata); + if (IS_ERR(xrcd)) { + ret = PTR_ERR(xrcd); + goto err; + } + xrcd->uobject = (cmd.fd == -1) ? uobj : NULL; + xrcd->inode = inode; + xrcd->device = file->device->ib_dev; + atomic_set(&xrcd->usecnt, 0); + new_xrcd = 1; } - xrcd->fd = cmd.fd; - xrcd->flags = cmd.oflags; - xrcd->uobject = uobj; - xrcd->device = file->device->ib_dev; - atomic_set(&xrcd->usecnt, 0); - uobj->object = xrcd; ret = idr_add_uobj(&ib_uverbs_xrc_domain_idr, uobj); if (ret) @@ -2375,6 +2503,18 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, memset(&resp, 0, sizeof resp); resp.xrcd_handle = uobj->id; + if (inode) { + if (new_xrcd) { + /* create new inode/xrcd table entry */ + ret = xrcd_table_insert(file->device->ib_dev, inode, xrcd); + if (ret) + goto err_insert_xrcd; + } + atomic_inc(&xrcd->usecnt); + } + if (f) + fput(f); + if (copy_to_user((void __user *) (unsigned long) cmd.response, &resp, sizeof resp)) { ret = -EFAULT; @@ -2389,16 +2529,31 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, up_write(&uobj->mutex); + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); return in_len; err_copy: - idr_remove_uobj(&ib_uverbs_pd_idr, uobj); + + if (inode) { + if (new_xrcd) + xrcd_table_delete(file->device->ib_dev, inode); + atomic_dec(&xrcd->usecnt); + } + +err_insert_xrcd: + idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); err_idr: ib_dealloc_xrcd(xrcd); err: put_uobj_write(uobj); + +err_table_mutex_unlock: + + if (f) + fput(f); + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); return ret; } @@ -2408,15 +2563,26 @@ ssize_t ib_uverbs_close_xrc_domain(struct ib_uverbs_file *file, { struct ib_uverbs_close_xrc_domain cmd; struct ib_uobject *uobj; - int ret; + struct ib_xrcd *xrcd = NULL; + struct inode *inode = NULL; + int ret = 0; if (copy_from_user(&cmd, buf, sizeof cmd)) return -EFAULT; + mutex_lock(&file->device->ib_dev->xrcd_table_mutex); uobj = idr_write_uobj(&ib_uverbs_xrc_domain_idr, cmd.xrcd_handle, file->ucontext); - if (!uobj) - return -EINVAL; + if (!uobj) { + ret = -EINVAL; + goto err_unlock_mutex; + } + + xrcd = (struct ib_xrcd *) (uobj->object); + inode = xrcd->inode; + + if (inode) + atomic_dec(&xrcd->usecnt); ret = ib_dealloc_xrcd(uobj->object); if (!ret) @@ -2424,8 +2590,11 @@ ssize_t ib_uverbs_close_xrc_domain(struct ib_uverbs_file *file, put_uobj_write(uobj); - if (ret) - return ret; + if (ret && !inode) + goto err_unlock_mutex; + + if (!ret && inode) + xrcd_table_delete(file->device->ib_dev, inode); idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); @@ -2435,6 +2604,27 @@ ssize_t ib_uverbs_close_xrc_domain(struct ib_uverbs_file *file, put_uobj(uobj); + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); return in_len; + +err_unlock_mutex: + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); + return ret; +} + +void ib_uverbs_dealloc_xrcd(struct ib_device *ib_dev, + struct ib_xrcd *xrcd) +{ + struct inode *inode = NULL; + int ret = 0; + + inode = xrcd->inode; + if (inode) + atomic_dec(&xrcd->usecnt); + + ret = ib_dealloc_xrcd(xrcd); + if (!ret && inode) + xrcd_table_delete(ib_dev, inode); } + diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index c3d1ffb..0f8c1b4 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -249,13 +249,15 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uobj); } + mutex_lock(&file->device->ib_dev->xrcd_table_mutex); list_for_each_entry_safe(uobj, tmp, &context->xrc_domain_list, list) { struct ib_xrcd *xrcd = uobj->object; idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); - ib_dealloc_xrcd(xrcd); + ib_uverbs_dealloc_xrcd(file->device->ib_dev, xrcd); kfree(uobj); } + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); list_for_each_entry_safe(uobj, tmp, &context->pd_list, list) { struct ib_pd *pd = uobj->object; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 85080ac..774b4e2 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -50,6 +50,8 @@ #include #include +#include +#include union ib_gid { u8 raw[16]; @@ -848,11 +850,8 @@ struct ib_pd { struct ib_xrcd { struct ib_device *device; struct ib_uobject *uobject; - struct rb_node node; - u32 xrc_domain_num; struct inode *inode; - int fd; - u32 flags; + struct rb_node node; atomic_t usecnt; /* count all resources */ }; @@ -1179,6 +1178,8 @@ struct ib_device { __be64 node_guid; u8 node_type; u8 phys_port_cnt; + struct rb_root ib_uverbs_xrcd_table; + struct mutex xrcd_table_mutex; }; struct ib_client { -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:45 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:45 +0300 Subject: [ofa-general] [PATCH 7/12 V2] core: kernel-space application XRC QPs Message-ID: <200807101851.46024.jackm@dev.mellanox.co.il> From b84af8925c44e3c40ecbbb9104f01f7adccf79be Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Tue, 8 Jul 2008 09:42:47 +0300 Subject: [PATCH] IB/core: implement kernel-space XRC. V2: no changes Signed-off-by: Jack Morgenstein --- drivers/infiniband/core/verbs.c | 56 +++++++++++++++++++++++++++++++++++++- include/rdma/ib_verbs.h | 34 +++++++++++++++++++++-- 2 files changed, 85 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 1b60806..41dddfa 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -244,6 +244,36 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd, } EXPORT_SYMBOL(ib_create_srq); +struct ib_srq *ib_create_xrc_srq(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *srq_init_attr) +{ + struct ib_srq *srq; + + if (!pd->device->create_xrc_srq) + return ERR_PTR(-ENOSYS); + + srq = pd->device->create_xrc_srq(pd, xrc_cq, xrcd, srq_init_attr, NULL); + + if (!IS_ERR(srq)) { + srq->device = pd->device; + srq->pd = pd; + srq->uobject = NULL; + srq->event_handler = srq_init_attr->event_handler; + srq->srq_context = srq_init_attr->srq_context; + srq->xrc_cq = xrc_cq; + srq->xrcd = xrcd; + atomic_inc(&pd->usecnt); + atomic_inc(&xrcd->usecnt); + atomic_inc(&xrc_cq->usecnt); + atomic_set(&srq->usecnt, 0); + } + + return srq; +} +EXPORT_SYMBOL(ib_create_xrc_srq); + int ib_modify_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr, enum ib_srq_attr_mask srq_attr_mask) @@ -308,12 +338,15 @@ struct ib_qp *ib_create_qp(struct ib_pd *pd, qp->event_handler = qp_init_attr->event_handler; qp->qp_context = qp_init_attr->qp_context; qp->qp_type = qp_init_attr->qp_type; - qp->xrcd = NULL; + qp->xrcd = qp->qp_type == IB_QPT_XRC ? + qp_init_attr->xrc_domain : NULL; atomic_inc(&pd->usecnt); atomic_inc(&qp_init_attr->send_cq->usecnt); atomic_inc(&qp_init_attr->recv_cq->usecnt); if (qp_init_attr->srq) atomic_inc(&qp_init_attr->srq->usecnt); + if (qp->qp_type == IB_QPT_XRC) + atomic_inc(&qp->xrcd->usecnt); } return qp; @@ -644,6 +677,7 @@ int ib_destroy_qp(struct ib_qp *qp) struct ib_cq *scq, *rcq; struct ib_srq *srq; struct ib_xrcd *xrcd; + enum ib_qp_type qp_type = qp->qp_type; int ret; pd = qp->pd; @@ -659,7 +693,7 @@ int ib_destroy_qp(struct ib_qp *qp) atomic_dec(&rcq->usecnt); if (srq) atomic_dec(&srq->usecnt); - if (xrcd) + if (qp_type == IB_QPT_XRC) atomic_dec(&xrcd->usecnt); } @@ -978,4 +1012,22 @@ int ib_dealloc_xrcd(struct ib_xrcd *xrcd) } EXPORT_SYMBOL(ib_dealloc_xrcd); +struct ib_xrcd *ib_alloc_xrcd(struct ib_device *device) +{ + struct ib_xrcd *xrcd; + + if (!device->alloc_xrcd) + return ERR_PTR(-ENOSYS); + + xrcd = device->alloc_xrcd(device, NULL, NULL); + if (!IS_ERR(xrcd)) { + xrcd->device = device; + xrcd->inode = NULL; + xrcd->uobject = NULL; + atomic_set(&xrcd->usecnt, 0); + } + return xrcd; +} +EXPORT_SYMBOL(ib_alloc_xrcd); + diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 774b4e2..cf8b594 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -758,6 +758,7 @@ struct ib_send_wr { u32 rkey; } fast_reg; } wr; + u32 xrc_remote_srq_num; /* valid for XRC sends only */ }; struct ib_recv_wr { @@ -883,6 +884,7 @@ struct ib_srq { void (*event_handler)(struct ib_event *, void *); void *srq_context; atomic_t usecnt; + u32 xrc_srq_num; }; struct ib_qp { @@ -1340,8 +1342,28 @@ int ib_query_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr); int ib_destroy_ah(struct ib_ah *ah); /** - * ib_create_srq - Creates a SRQ associated with the specified protection - * domain. + * ib_create_xrc_srq - Creates an XRC SRQ associated with the specified + * protection domain, cq, and xrc domain. + * @pd: The protection domain associated with the SRQ. + * @xrc_cq: The cq to be associated with the XRC SRQ. + * @xrcd: The XRC domain to be associated with the XRC SRQ. + * @srq_init_attr: A list of initial attributes required to create the + * XRC SRQ. If XRC SRQ creation succeeds, then the attributes are updated + * to the actual capabilities of the created XRC SRQ. + * + * srq_attr->max_wr and srq_attr->max_sge are read the determine the + * requested size of the XRC SRQ, and set to the actual values allocated + * on return. If ib_create_xrc_srq() succeeds, then max_wr and max_sge + * will always be at least as large as the requested values. + */ +struct ib_srq *ib_create_xrc_srq(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *srq_init_attr); + +/** + * ib_create_srq - Creates an SRQ associated with the specified + * protection domain. * @pd: The protection domain associated with the SRQ. * @srq_init_attr: A list of initial attributes required to create the * SRQ. If SRQ creation succeeds, then the attributes are updated to @@ -2062,8 +2084,14 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid); /** * ib_dealloc_xrcd - Deallocates an extended reliably connected domain. - * @pd: The xrc domain to deallocate. + * @xrcd: The xrc domain to deallocate. */ int ib_dealloc_xrcd(struct ib_xrcd *xrcd); +/** + * ib_alloc_xrcd - Allocates an extended reliably connected domain. + * @device: The device on which to allocate the xrcd. + */ +struct ib_xrcd *ib_alloc_xrcd(struct ib_device *device); + #endif /* IB_VERBS_H */ -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:52:03 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:52:03 +0300 Subject: [ofa-general] [PATCH 10/12 V2] mlx4: XRC kernel-space app QP support Message-ID: <200807101852.03313.jackm@dev.mellanox.co.il> From c734d90a3d28f186d599d12d6d12ff1b3a09fd22 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Mon, 7 Jul 2008 11:47:21 +0300 Subject: [PATCH] mlx4: Implement kernel-space XRC. V2: no changes. Signed-off-by: Jack Morgenstein --- drivers/infiniband/hw/mlx4/cq.c | 37 ++++++++++++++++++++++++++++++++++--- drivers/infiniband/hw/mlx4/qp.c | 8 +++++++- drivers/infiniband/hw/mlx4/srq.c | 16 +++++++++++++++- drivers/net/mlx4/mlx4.h | 1 - drivers/net/mlx4/srq.c | 26 +++++++++++++++++++------- include/linux/mlx4/device.h | 1 + include/linux/mlx4/srq.h | 12 ++++++++++++ 7 files changed, 88 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 299f208..27ce6b6 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -32,6 +32,7 @@ #include #include +#include #include "mlx4_ib.h" #include "user.h" @@ -536,9 +537,11 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, struct mlx4_qp *mqp; struct mlx4_ib_wq *wq; struct mlx4_ib_srq *srq; + struct mlx4_srq *msrq; int is_send; int is_error; u32 g_mlpath_rqpn; + int is_xrc_recv = 0; u16 wqe_ctr; repoll: @@ -580,7 +583,24 @@ repoll: goto repoll; } - if (!*cur_qp || + if ((be32_to_cpu(cqe->my_qpn) & (1 << 23)) && !is_send) { + /* + * We do not have to take the XRC SRQ table lock here, + * because CQs will be locked while XRC SRQs are removed + * from the table. + */ + msrq = __mlx4_srq_lookup(to_mdev(cq->ibcq.device)->dev, + be32_to_cpu(cqe->g_mlpath_rqpn) & + 0xffffff); + if (unlikely(!msrq)) { + printk(KERN_WARNING "CQ %06x with entry for unknown " + "XRC SRQ %06x\n", cq->mcq.cqn, + be32_to_cpu(cqe->g_mlpath_rqpn) & 0xffffff); + return -EINVAL; + } + is_xrc_recv = 1; + srq = to_mibsrq(msrq); + } else if (!*cur_qp || (be32_to_cpu(cqe->my_qpn) & 0xffffff) != (*cur_qp)->mqp.qpn) { /* * We do not have to take the QP table lock here, @@ -598,7 +618,7 @@ repoll: *cur_qp = to_mibqp(mqp); } - wc->qp = &(*cur_qp)->ibqp; + wc->qp = is_xrc_recv ? NULL: &(*cur_qp)->ibqp; if (is_send) { wq = &(*cur_qp)->sq; @@ -608,6 +628,10 @@ repoll: } wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)]; ++wq->tail; + } else if (is_xrc_recv) { + wqe_ctr = be16_to_cpu(cqe->wqe_index); + wc->wr_id = srq->wrid[wqe_ctr]; + mlx4_ib_free_srq_wqe(srq, wqe_ctr); } else if ((*cur_qp)->ibqp.srq) { srq = to_msrq((*cur_qp)->ibqp.srq); wqe_ctr = be16_to_cpu(cqe->wqe_index); @@ -736,6 +760,10 @@ void __mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq) int nfreed = 0; struct mlx4_cqe *cqe, *dest; u8 owner_bit; + int is_xrc_srq = 0; + + if (srq && srq->ibsrq.xrc_cq) + is_xrc_srq = 1; /* * First we need to find the current producer index, so we @@ -754,7 +782,10 @@ void __mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq) */ while ((int) --prod_index - (int) cq->mcq.cons_index >= 0) { cqe = get_cqe(cq, prod_index & cq->ibcq.cqe); - if ((be32_to_cpu(cqe->my_qpn) & 0xffffff) == qpn) { + if (((be32_to_cpu(cqe->my_qpn) & 0xffffff) == qpn) || + (is_xrc_srq && + (be32_to_cpu(cqe->g_mlpath_rqpn) & 0xffffff) == + srq->msrq.srqn)) { if (srq && !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) mlx4_ib_free_srq_wqe(srq, be16_to_cpu(cqe->wqe_index)); ++nfreed; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index f37a69c..a16f099 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -261,6 +261,7 @@ static int send_wqe_overhead(enum ib_qp_type type, u32 flags) case IB_QPT_UC: return sizeof (struct mlx4_wqe_ctrl_seg) + sizeof (struct mlx4_wqe_raddr_seg); + case IB_QPT_XRC: case IB_QPT_RC: return sizeof (struct mlx4_wqe_ctrl_seg) + sizeof (struct mlx4_wqe_atomic_seg) + @@ -1507,6 +1508,10 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, size = sizeof *ctrl / 16; switch (ibqp->qp_type) { + case IB_QPT_XRC: + ctrl->srcrb_flags |= + cpu_to_be32(wr->xrc_remote_srq_num << 8); + /* fall thru */ case IB_QPT_RC: case IB_QPT_UC: switch (wr->opcode) { @@ -1821,7 +1826,8 @@ int mlx4_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr qp_attr->qp_access_flags = to_ib_qp_access_flags(be32_to_cpu(context.params2)); - if (qp->ibqp.qp_type == IB_QPT_RC || qp->ibqp.qp_type == IB_QPT_UC) { + if (qp->ibqp.qp_type == IB_QPT_RC || qp->ibqp.qp_type == IB_QPT_UC || + qp->ibqp.qp_type == IB_QPT_XRC) { to_ib_ah_attr(dev->dev, &qp_attr->ah_attr, &context.pri_path); to_ib_ah_attr(dev->dev, &qp_attr->alt_ah_attr, &context.alt_path); qp_attr->alt_pkey_index = context.alt_path.pkey_index & 0x7f; diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c index 55dca71..dfc9338 100644 --- a/drivers/infiniband/hw/mlx4/srq.c +++ b/drivers/infiniband/hw/mlx4/srq.c @@ -181,11 +181,13 @@ struct ib_srq *mlx4_ib_create_xrc_srq(struct ib_pd *pd, srq->msrq.event = mlx4_ib_srq_event; - if (pd->uobject) + if (pd->uobject) { if (ib_copy_to_udata(udata, &srq->msrq.srqn, sizeof (__u32))) { err = -EFAULT; goto err_wrid; } + } else + srq->ibsrq.xrc_srq_num = srq->msrq.srqn; init_attr->attr.max_wr = srq->msrq.max - 1; @@ -271,6 +273,18 @@ int mlx4_ib_destroy_srq(struct ib_srq *srq) { struct mlx4_ib_dev *dev = to_mdev(srq->device); struct mlx4_ib_srq *msrq = to_msrq(srq); + struct mlx4_ib_cq *cq; + + mlx4_srq_invalidate(dev->dev, &msrq->msrq); + + if (srq->xrc_cq && !srq->uobject) { + cq = to_mcq(srq->xrc_cq); + spin_lock_irq(&cq->lock); + __mlx4_ib_cq_clean(cq, -1, msrq); + mlx4_srq_remove(dev->dev, &msrq->msrq); + spin_unlock_irq(&cq->lock); + } else + mlx4_srq_remove(dev->dev, &msrq->msrq); mlx4_srq_free(dev->dev, &msrq->msrq); mlx4_mtt_cleanup(dev->dev, &msrq->mtt); diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 82b3273..426428d 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -221,7 +221,6 @@ struct mlx4_eq_table { struct mlx4_srq_table { struct mlx4_bitmap bitmap; spinlock_t lock; - struct radix_tree_root tree; struct mlx4_icm_table table; struct mlx4_icm_table cmpt_table; }; diff --git a/drivers/net/mlx4/srq.c b/drivers/net/mlx4/srq.c index 585a3cf..9d4d9d2 100644 --- a/drivers/net/mlx4/srq.c +++ b/drivers/net/mlx4/srq.c @@ -64,7 +64,8 @@ void mlx4_srq_event(struct mlx4_dev *dev, u32 srqn, int event_type) spin_lock(&srq_table->lock); - srq = radix_tree_lookup(&srq_table->tree, srqn & (dev->caps.num_srqs - 1)); + srq = radix_tree_lookup(&dev->srq_table_tree, + srqn & (dev->caps.num_srqs - 1)); if (srq) atomic_inc(&srq->refcount); @@ -131,7 +132,7 @@ int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, u32 cqn, u16 xrcd, goto err_put; spin_lock_irq(&srq_table->lock); - err = radix_tree_insert(&srq_table->tree, srq->srqn, srq); + err = radix_tree_insert(&dev->srq_table_tree, srq->srqn, srq); spin_unlock_irq(&srq_table->lock); if (err) goto err_cmpt_put; @@ -170,7 +171,7 @@ int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, u32 cqn, u16 xrcd, err_radix: spin_lock_irq(&srq_table->lock); - radix_tree_delete(&srq_table->tree, srq->srqn); + radix_tree_delete(&dev->srq_table_tree, srq->srqn); spin_unlock_irq(&srq_table->lock); err_cmpt_put: @@ -186,18 +187,29 @@ err_out: } EXPORT_SYMBOL_GPL(mlx4_srq_alloc); -void mlx4_srq_free(struct mlx4_dev *dev, struct mlx4_srq *srq) +void mlx4_srq_invalidate(struct mlx4_dev *dev, struct mlx4_srq *srq) { - struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table; int err; err = mlx4_HW2SW_SRQ(dev, NULL, srq->srqn); if (err) mlx4_warn(dev, "HW2SW_SRQ failed (%d) for SRQN %06x\n", err, srq->srqn); +} +EXPORT_SYMBOL_GPL(mlx4_srq_invalidate); + +void mlx4_srq_remove(struct mlx4_dev *dev, struct mlx4_srq *srq) +{ + struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table; spin_lock_irq(&srq_table->lock); - radix_tree_delete(&srq_table->tree, srq->srqn); + radix_tree_delete(&dev->srq_table_tree, srq->srqn); spin_unlock_irq(&srq_table->lock); +} +EXPORT_SYMBOL_GPL(mlx4_srq_remove); + +void mlx4_srq_free(struct mlx4_dev *dev, struct mlx4_srq *srq) +{ + struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table; if (atomic_dec_and_test(&srq->refcount)) complete(&srq->free); @@ -243,7 +255,7 @@ int mlx4_init_srq_table(struct mlx4_dev *dev) int err; spin_lock_init(&srq_table->lock); - INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); + INIT_RADIX_TREE(&dev->srq_table_tree, GFP_ATOMIC); err = mlx4_bitmap_init(&srq_table->bitmap, dev->caps.num_srqs, dev->caps.num_srqs - 1, dev->caps.reserved_srqs); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index c812a78..a08c56f 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -326,6 +326,7 @@ struct mlx4_dev { unsigned long flags; struct mlx4_caps caps; struct radix_tree_root qp_table_tree; + struct radix_tree_root srq_table_tree; u32 rev_id; char board_id[MLX4_BOARD_ID_LEN]; }; diff --git a/include/linux/mlx4/srq.h b/include/linux/mlx4/srq.h index 799a069..5e041e5 100644 --- a/include/linux/mlx4/srq.h +++ b/include/linux/mlx4/srq.h @@ -33,10 +33,22 @@ #ifndef MLX4_SRQ_H #define MLX4_SRQ_H +#include +#include + struct mlx4_wqe_srq_next_seg { u16 reserved1; __be16 next_wqe_index; u32 reserved2[3]; }; +void mlx4_srq_invalidate(struct mlx4_dev *dev, struct mlx4_srq *srq); +void mlx4_srq_remove(struct mlx4_dev *dev, struct mlx4_srq *srq); + +static inline struct mlx4_srq *__mlx4_srq_lookup(struct mlx4_dev *dev, u32 srqn) +{ + return radix_tree_lookup(&dev->srq_table_tree, + srqn & (dev->caps.num_srqs - 1)); +} + #endif /* MLX4_SRQ_H */ -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:52:06 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:52:06 +0300 Subject: [ofa-general] [PATCH 11/12 V2] mlx4: XRC receive-only QPs Message-ID: <200807101852.06851.jackm@dev.mellanox.co.il> From 7e94b2921f92165b7c6d2f36cfdda6ffb36799d3 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Mon, 7 Jul 2008 17:47:58 +0300 Subject: [PATCH] mlx4: Implement XRC receive-only QP support. Support for XRC RCV-only QP (requested by userspace, but resides in kernel space). V2: lots of cleanups. 1. kzalloc instead of memset 2. eliminated use of ib/core qp create flag for xrc rcv qp. 3. checkpatch cleanups. Signed-off-by: Jack Morgenstein --- drivers/infiniband/hw/mlx4/cq.c | 2 +- drivers/infiniband/hw/mlx4/main.c | 62 ++++++- drivers/infiniband/hw/mlx4/mlx4_ib.h | 15 ++ drivers/infiniband/hw/mlx4/qp.c | 301 +++++++++++++++++++++++++++++++++- 4 files changed, 367 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 27ce6b6..c0b1aae 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -174,7 +174,7 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector if (entries < 1 || entries > dev->dev->caps.max_cqes) return ERR_PTR(-EINVAL); - cq = kmalloc(sizeof *cq, GFP_KERNEL); + cq = kzalloc(sizeof *cq, GFP_KERNEL); if (!cq) return ERR_PTR(-ENOMEM); diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 936be03..030f696 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -409,7 +409,7 @@ static struct ib_pd *mlx4_ib_alloc_pd(struct ib_device *ibdev, struct mlx4_ib_pd *pd; int err; - pd = kmalloc(sizeof *pd, GFP_KERNEL); + pd = kzalloc(sizeof *pd, GFP_KERNEL); if (!pd) return ERR_PTR(-ENOMEM); @@ -451,12 +451,18 @@ static int mlx4_ib_mcg_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid) &to_mqp(ibqp)->mqp, gid->raw); } +static void mlx4_dummy_comp_handler(struct ib_cq *cq, void *cq_context) +{ +} + static struct ib_xrcd *mlx4_ib_alloc_xrcd(struct ib_device *ibdev, struct ib_ucontext *context, struct ib_udata *udata) { struct mlx4_ib_xrcd *xrcd; struct mlx4_ib_dev *mdev = to_mdev(ibdev); + struct ib_pd *pd; + struct ib_cq *cq; int err; if (!(mdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC)) @@ -467,23 +473,51 @@ static struct ib_xrcd *mlx4_ib_alloc_xrcd(struct ib_device *ibdev, return ERR_PTR(-ENOMEM); err = mlx4_xrcd_alloc(mdev->dev, &xrcd->xrcdn); - if (err) { - kfree(xrcd); - return ERR_PTR(err); + if (err) + goto err_xrcd; + + pd = mlx4_ib_alloc_pd(ibdev, NULL, NULL); + if (IS_ERR(pd)) { + err = PTR_ERR(pd); + goto err_pd; } + pd->device = ibdev; + + cq = mlx4_ib_create_cq(ibdev, 1, 0, NULL, NULL); + if (IS_ERR(cq)) { + err = PTR_ERR(cq); + goto err_cq; + } + cq->device = ibdev; + cq->comp_handler = mlx4_dummy_comp_handler; if (context) if (ib_copy_to_udata(udata, &xrcd->xrcdn, sizeof(__u32))) { - mlx4_xrcd_free(mdev->dev, xrcd->xrcdn); - kfree(xrcd); - return ERR_PTR(-EFAULT); + err = -EFAULT; + goto err_copy; } + xrcd->cq = cq; + xrcd->pd = pd; return &xrcd->ibxrcd; + +err_copy: + mlx4_ib_destroy_cq(cq); +err_cq: + mlx4_ib_dealloc_pd(pd); +err_pd: + mlx4_xrcd_free(mdev->dev, xrcd->xrcdn); +err_xrcd: + kfree(xrcd); + return ERR_PTR(err); } static int mlx4_ib_dealloc_xrcd(struct ib_xrcd *xrcd) { + struct mlx4_ib_xrcd *mxrcd = to_mxrcd(xrcd); + + mlx4_ib_destroy_cq(mxrcd->cq); + mlx4_ib_dealloc_pd(mxrcd->pd); mlx4_xrcd_free(to_mdev(xrcd->device)->dev, to_mxrcd(xrcd)->xrcdn); kfree(xrcd); @@ -681,18 +715,28 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.create_xrc_srq = mlx4_ib_create_xrc_srq; ibdev->ib_dev.alloc_xrcd = mlx4_ib_alloc_xrcd; ibdev->ib_dev.dealloc_xrcd = mlx4_ib_dealloc_xrcd; + ibdev->ib_dev.create_xrc_rcv_qp = mlx4_ib_create_xrc_rcv_qp; + ibdev->ib_dev.modify_xrc_rcv_qp = mlx4_ib_modify_xrc_rcv_qp; + ibdev->ib_dev.query_xrc_rcv_qp = mlx4_ib_query_xrc_rcv_qp; + ibdev->ib_dev.reg_xrc_rcv_qp = mlx4_ib_reg_xrc_rcv_qp; + ibdev->ib_dev.unreg_xrc_rcv_qp = mlx4_ib_unreg_xrc_rcv_qp; ibdev->ib_dev.uverbs_cmd_mask |= (1ull << IB_USER_VERBS_CMD_CREATE_XRC_SRQ) | (1ull << IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN) | - (1ull << IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN); + (1ull << IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN) | + (1ull << IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP) | + (1ull << IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP) | + (1ull << IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP) | + (1ull << IB_USER_VERBS_CMD_REG_XRC_RCV_QP) | + (1ull << IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP); } - if (init_node_data(ibdev)) goto err_map; spin_lock_init(&ibdev->sm_lock); mutex_init(&ibdev->cap_mask_mutex); + mutex_init(&ibdev->xrc_reg_mutex); if (ib_register_device(&ibdev->ib_dev)) goto err_map; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 4c60c2f..7bf9a46 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -58,6 +58,8 @@ struct mlx4_ib_pd { struct mlx4_ib_xrcd { struct ib_xrcd ibxrcd; u32 xrcdn; + struct ib_pd *pd; + struct ib_cq *cq; }; struct mlx4_ib_cq_buf { @@ -108,6 +110,7 @@ struct mlx4_ib_wq { enum mlx4_ib_qp_flags { MLX4_IB_QP_LSO = 1 << 0, MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + MLX4_IB_XRC_RCV = 1 << 2, }; struct mlx4_ib_qp { @@ -130,6 +133,7 @@ struct mlx4_ib_qp { int buf_size; struct mutex mutex; u32 flags; + struct list_head xrc_reg_list; u16 xrcdn; u8 port; u8 alt_port; @@ -173,6 +177,7 @@ struct mlx4_ib_dev { spinlock_t sm_lock; struct mutex cap_mask_mutex; + struct mutex xrc_reg_mutex; }; static inline struct mlx4_ib_dev *to_mdev(struct ib_device *ibdev) @@ -310,6 +315,16 @@ int mlx4_ib_map_phys_fmr(struct ib_fmr *ibfmr, u64 *page_list, int npages, u64 iova); int mlx4_ib_unmap_fmr(struct list_head *fmr_list); int mlx4_ib_fmr_dealloc(struct ib_fmr *fmr); +int mlx4_ib_create_xrc_rcv_qp(struct ib_qp_init_attr *init_attr, + u32 *qp_num); +int mlx4_ib_modify_xrc_rcv_qp(struct ib_xrcd *xrcd, u32 qp_num, + struct ib_qp_attr *attr, int attr_mask); +int mlx4_ib_query_xrc_rcv_qp(struct ib_xrcd *xrcd, u32 qp_num, + struct ib_qp_attr *attr, int attr_mask, + struct ib_qp_init_attr *init_attr); +int mlx4_ib_reg_xrc_rcv_qp(struct ib_xrcd *xrcd, void *context, u32 qp_num); +int mlx4_ib_unreg_xrc_rcv_qp(struct ib_xrcd *xrcd, void *context, u32 qp_num); + static inline int mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) { diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index a16f099..24c904f 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -56,6 +56,12 @@ enum { MLX4_IB_UD_HEADER_SIZE = 72 }; + +struct mlx4_ib_xrc_reg_entry { + struct list_head list; + void *context; +}; + struct mlx4_ib_sqp { struct mlx4_ib_qp qp; int pkey_index; @@ -202,14 +208,15 @@ static inline unsigned pad_wraparound(struct mlx4_ib_qp *qp, int ind) static void mlx4_ib_qp_event(struct mlx4_qp *qp, enum mlx4_event type) { struct ib_event event; - struct ib_qp *ibqp = &to_mibqp(qp)->ibqp; + struct mlx4_ib_qp *mqp = to_mibqp(qp); + struct ib_qp *ibqp = &mqp->ibqp; + struct mlx4_ib_xrc_reg_entry *ctx_entry; if (type == MLX4_EVENT_TYPE_PATH_MIG) to_mibqp(qp)->port = to_mibqp(qp)->alt_port; if (ibqp->event_handler) { event.device = ibqp->device; - event.element.qp = ibqp; switch (type) { case MLX4_EVENT_TYPE_PATH_MIG: event.event = IB_EVENT_PATH_MIG; @@ -241,6 +248,15 @@ static void mlx4_ib_qp_event(struct mlx4_qp *qp, enum mlx4_event type) return; } + if (unlikely(ibqp->qp_type == IB_QPT_XRC && + mqp->flags & MLX4_IB_XRC_RCV)) { + event.event |= IB_XRC_QP_EVENT_FLAG; + event.element.xrc_qp_num = ibqp->qp_num; + list_for_each_entry(ctx_entry, &mqp->xrc_reg_list, list) + ibqp->event_handler(&event, ctx_entry->context); + return; + } + event.element.qp = ibqp; ibqp->event_handler(&event, ibqp->qp_context); } } @@ -712,7 +728,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, case IB_QPT_UC: case IB_QPT_UD: { - qp = kmalloc(sizeof *qp, GFP_KERNEL); + qp = kzalloc(sizeof *qp, GFP_KERNEL); if (!qp) return ERR_PTR(-ENOMEM); @@ -1887,3 +1903,282 @@ out: return err; } +int mlx4_ib_create_xrc_rcv_qp(struct ib_qp_init_attr *init_attr, + u32 *qp_num) +{ + struct mlx4_ib_dev *dev = to_mdev(init_attr->xrc_domain->device); + struct mlx4_ib_xrcd *xrcd = to_mxrcd(init_attr->xrc_domain); + struct mlx4_ib_qp *qp; + struct ib_qp *ibqp; + struct mlx4_ib_xrc_reg_entry *ctx_entry; + int err; + + if (!(dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC)) + return -ENOSYS; + + if (init_attr->qp_type != IB_QPT_XRC) + return -EINVAL; + + ctx_entry = kmalloc(sizeof *ctx_entry, GFP_KERNEL); + if (!ctx_entry) + return -ENOMEM; + + qp = kzalloc(sizeof *qp, GFP_KERNEL); + if (!qp) { + kfree(ctx_entry); + return -ENOMEM; + } + qp->flags = MLX4_IB_XRC_RCV; + qp->xrcdn = to_mxrcd(init_attr->xrc_domain)->xrcdn; + INIT_LIST_HEAD(&qp->xrc_reg_list); + err = create_qp_common(dev, xrcd->pd, init_attr, NULL, 0, qp); + if (err) { + kfree(ctx_entry); + kfree(qp); + return err; + } + + ibqp = &qp->ibqp; + /* set the ibpq attributes which will be used by the mlx4 module */ + ibqp->qp_num = qp->mqp.qpn; + ibqp->device = init_attr->xrc_domain->device; + ibqp->pd = xrcd->pd; + ibqp->send_cq = ibqp->recv_cq = xrcd->cq; + ibqp->event_handler = init_attr->event_handler; + ibqp->qp_context = init_attr->qp_context; + ibqp->qp_type = init_attr->qp_type; + ibqp->xrcd = init_attr->xrc_domain; + + mutex_lock(&qp->mutex); + ctx_entry->context = init_attr->qp_context; + list_add_tail(&ctx_entry->list, &qp->xrc_reg_list); + mutex_unlock(&qp->mutex); + *qp_num = qp->mqp.qpn; + return 0; +} + +int mlx4_ib_modify_xrc_rcv_qp(struct ib_xrcd *ibxrcd, u32 qp_num, + struct ib_qp_attr *attr, int attr_mask) +{ + struct mlx4_ib_dev *dev = to_mdev(ibxrcd->device); + struct mlx4_ib_xrcd *xrcd = to_mxrcd(ibxrcd); + struct mlx4_qp *mqp; + int err = -EINVAL; + + if (!(dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC)) + return -ENOSYS; + + mutex_lock(&dev->xrc_reg_mutex); + mqp = __mlx4_qp_lookup(dev->dev, qp_num); + if (unlikely(!mqp)) { + printk(KERN_WARNING "mlx4_ib_reg_xrc_rcv_qp: " + "unknown QPN %06x\n", qp_num); + goto err_out; + } + + if (xrcd->xrcdn != to_mxrcd(to_mibqp(mqp)->ibqp.xrcd)->xrcdn) + goto err_out; + + err = mlx4_ib_modify_qp(&(to_mibqp(mqp)->ibqp), attr, attr_mask, NULL); + mutex_unlock(&dev->xrc_reg_mutex); + return err; + +err_out: + mutex_unlock(&dev->xrc_reg_mutex); + return err; +} + +int mlx4_ib_query_xrc_rcv_qp(struct ib_xrcd *ibxrcd, u32 qp_num, + struct ib_qp_attr *qp_attr, int qp_attr_mask, + struct ib_qp_init_attr *qp_init_attr) +{ + struct mlx4_ib_dev *dev = to_mdev(ibxrcd->device); + struct mlx4_ib_xrcd *xrcd = to_mxrcd(ibxrcd); + struct mlx4_ib_qp *qp; + struct mlx4_qp *mqp; + struct mlx4_qp_context context; + int mlx4_state; + int err = -EINVAL; + + if (!(dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC)) + return -ENOSYS; + + mutex_lock(&dev->xrc_reg_mutex); + mqp = __mlx4_qp_lookup(dev->dev, qp_num); + if (unlikely(!mqp)) { + printk(KERN_WARNING "mlx4_ib_reg_xrc_rcv_qp: " + "unknown QPN %06x\n", qp_num); + goto err_out; + } + + qp = to_mibqp(mqp); + if (xrcd->xrcdn != to_mxrcd(qp->ibqp.xrcd)->xrcdn) + goto err_out; + + if (qp->state == IB_QPS_RESET) { + qp_attr->qp_state = IB_QPS_RESET; + goto done; + } + + err = mlx4_qp_query(dev->dev, mqp, &context); + if (err) + goto err_out; + + mlx4_state = be32_to_cpu(context.flags) >> 28; + + qp_attr->qp_state = to_ib_qp_state(mlx4_state); + qp_attr->path_mtu = context.mtu_msgmax >> 5; + qp_attr->path_mig_state = + to_ib_mig_state((be32_to_cpu(context.flags) >> 11) & 0x3); + qp_attr->qkey = be32_to_cpu(context.qkey); + qp_attr->rq_psn = be32_to_cpu(context.rnr_nextrecvpsn) & 0xffffff; + qp_attr->sq_psn = be32_to_cpu(context.next_send_psn) & 0xffffff; + qp_attr->dest_qp_num = be32_to_cpu(context.remote_qpn) & 0xffffff; + qp_attr->qp_access_flags = + to_ib_qp_access_flags(be32_to_cpu(context.params2)); + + if (qp->ibqp.qp_type == IB_QPT_RC || qp->ibqp.qp_type == IB_QPT_UC || + qp->ibqp.qp_type == IB_QPT_XRC) { + to_ib_ah_attr(dev->dev, &qp_attr->ah_attr, &context.pri_path); + to_ib_ah_attr(dev->dev, &qp_attr->alt_ah_attr, + &context.alt_path); + qp_attr->alt_pkey_index = context.alt_path.pkey_index & 0x7f; + qp_attr->alt_port_num = qp_attr->alt_ah_attr.port_num; + } + + qp_attr->pkey_index = context.pri_path.pkey_index & 0x7f; + if (qp_attr->qp_state == IB_QPS_INIT) + qp_attr->port_num = qp->port; + else + qp_attr->port_num = context.pri_path.sched_queue & 0x40 ? 2 : 1; + + /* qp_attr->en_sqd_async_notify is only applicable in modify qp */ + qp_attr->sq_draining = mlx4_state == MLX4_QP_STATE_SQ_DRAINING; + + qp_attr->max_rd_atomic = + 1 << ((be32_to_cpu(context.params1) >> 21) & 0x7); + + qp_attr->max_dest_rd_atomic = + 1 << ((be32_to_cpu(context.params2) >> 21) & 0x7); + qp_attr->min_rnr_timer = + (be32_to_cpu(context.rnr_nextrecvpsn) >> 24) & 0x1f; + qp_attr->timeout = context.pri_path.ackto >> 3; + qp_attr->retry_cnt = (be32_to_cpu(context.params1) >> 16) & 0x7; + qp_attr->rnr_retry = (be32_to_cpu(context.params1) >> 13) & 0x7; + qp_attr->alt_timeout = context.alt_path.ackto >> 3; + +done: + qp_attr->cur_qp_state = qp_attr->qp_state; + qp_attr->cap.max_recv_wr = 0; + qp_attr->cap.max_recv_sge = 0; + qp_attr->cap.max_send_wr = 0; + qp_attr->cap.max_send_sge = 0; + qp_attr->cap.max_inline_data = 0; + qp_init_attr->cap = qp_attr->cap; + + mutex_unlock(&dev->xrc_reg_mutex); + return 0; + +err_out: + mutex_unlock(&dev->xrc_reg_mutex); + return err; +} + +int mlx4_ib_reg_xrc_rcv_qp(struct ib_xrcd *xrcd, void *context, u32 qp_num) +{ + + struct mlx4_ib_xrcd *mxrcd = to_mxrcd(xrcd); + + struct mlx4_qp *mqp; + struct mlx4_ib_qp *mibqp; + struct mlx4_ib_xrc_reg_entry *ctx_entry, *tmp; + int err = -EINVAL; + + mutex_lock(&to_mdev(xrcd->device)->xrc_reg_mutex); + mqp = __mlx4_qp_lookup(to_mdev(xrcd->device)->dev, qp_num); + if (unlikely(!mqp)) { + printk(KERN_WARNING "mlx4_ib_reg_xrc_rcv_qp: " + "unknown QPN %06x\n", qp_num); + goto err_out; + } + + mibqp = to_mibqp(mqp); + + if (mxrcd->xrcdn != to_mxrcd(mibqp->ibqp.xrcd)->xrcdn) + goto err_out; + + ctx_entry = kmalloc(sizeof *ctx_entry, GFP_KERNEL); + if (!ctx_entry) { + err = -ENOMEM; + goto err_out; + } + + mutex_lock(&mibqp->mutex); + list_for_each_entry(tmp, &mibqp->xrc_reg_list, list) + if (tmp->context == context) { + mutex_unlock(&mibqp->mutex); + kfree(ctx_entry); + mutex_unlock(&to_mdev(xrcd->device)->xrc_reg_mutex); + return 0; + } + + ctx_entry->context = context; + list_add_tail(&ctx_entry->list, &mibqp->xrc_reg_list); + mutex_unlock(&mibqp->mutex); + mutex_unlock(&to_mdev(xrcd->device)->xrc_reg_mutex); + return 0; + +err_out: + mutex_unlock(&to_mdev(xrcd->device)->xrc_reg_mutex); + return err; +} + +int mlx4_ib_unreg_xrc_rcv_qp(struct ib_xrcd *xrcd, void *context, u32 qp_num) +{ + + struct mlx4_ib_xrcd *mxrcd = to_mxrcd(xrcd); + + struct mlx4_qp *mqp; + struct mlx4_ib_qp *mibqp; + struct mlx4_ib_xrc_reg_entry *ctx_entry, *tmp; + int found = 0; + int err = -EINVAL; + + mutex_lock(&to_mdev(xrcd->device)->xrc_reg_mutex); + mqp = __mlx4_qp_lookup(to_mdev(xrcd->device)->dev, qp_num); + if (unlikely(!mqp)) { + printk(KERN_WARNING "mlx4_ib_unreg_xrc_rcv_qp: " + "unknown QPN %06x\n", qp_num); + goto err_out; + } + + mibqp = to_mibqp(mqp); + + if (mxrcd->xrcdn != (mibqp->xrcdn & 0xffff)) + goto err_out; + + mutex_lock(&mibqp->mutex); + list_for_each_entry_safe(ctx_entry, tmp, &mibqp->xrc_reg_list, list) + if (ctx_entry->context == context) { + found = 1; + list_del(&ctx_entry->list); + kfree(ctx_entry); + break; + } + + mutex_unlock(&mibqp->mutex); + if (!found) + goto err_out; + + /* destroy the QP if the registration list is empty */ + if (list_empty(&mibqp->xrc_reg_list)) + mlx4_ib_destroy_qp(&mibqp->ibqp); + + mutex_unlock(&to_mdev(xrcd->device)->xrc_reg_mutex); + return 0; + +err_out: + mutex_unlock(&to_mdev(xrcd->device)->xrc_reg_mutex); + return err; +} + -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:52:16 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:52:16 +0300 Subject: [ofa-general] [PATCH 12/12] libibverbs: XRC man pages Message-ID: <200807101852.16738.jackm@dev.mellanox.co.il> From 02cf3664488c612af01ea39cca1cb776ecd1d2e8 Mon Sep 17 00:00:00 2001 From: Barak Date: Wed, 9 Jul 2008 13:06:05 +0300 Subject: [PATCH] Add XRC (eXtended Reliable Connection) support to all of the relevant man pages and add new man pages for new XRC verbs. Signed-off-by: Dotan Barak Signed-off-by: Jack Morgenstein --- Makefile.am | 15 ++++- man/ibv_create_qp.3 | 3 +- man/ibv_create_srq.3 | 14 ++++ man/ibv_create_xrc_rcv_qp.3 | 70 +++++++++++++++++++++ man/ibv_modify_xrc_rcv_qp.3 | 141 +++++++++++++++++++++++++++++++++++++++++++ man/ibv_open_xrc_domain.3 | 80 ++++++++++++++++++++++++ man/ibv_post_send.3 | 20 ++++--- man/ibv_query_xrc_rcv_qp.3 | 89 +++++++++++++++++++++++++++ man/ibv_reg_xrc_rcv_qp.3 | 57 +++++++++++++++++ 9 files changed, 476 insertions(+), 13 deletions(-) create mode 100644 man/ibv_create_xrc_rcv_qp.3 create mode 100644 man/ibv_modify_xrc_rcv_qp.3 create mode 100644 man/ibv_open_xrc_domain.3 create mode 100644 man/ibv_query_xrc_rcv_qp.3 create mode 100644 man/ibv_reg_xrc_rcv_qp.3 diff --git a/Makefile.am b/Makefile.am index 9b05306..8aeff67 100644 --- a/Makefile.am +++ b/Makefile.am @@ -44,15 +44,18 @@ man_MANS = man/ibv_asyncwatch.1 man/ibv_devices.1 man/ibv_devinfo.1 \ man/ibv_srq_pingpong.1 man/ibv_alloc_pd.3 man/ibv_attach_mcast.3 \ man/ibv_create_ah.3 man/ibv_create_ah_from_wc.3 \ man/ibv_create_comp_channel.3 man/ibv_create_cq.3 \ - man/ibv_create_qp.3 man/ibv_create_srq.3 man/ibv_event_type_str.3 \ + man/ibv_create_qp.3 man/ibv_create_srq.3 \ + man/ibv_create_xrc_rcv_qp.3 man/ibv_event_type_str.3 \ man/ibv_fork_init.3 man/ibv_get_async_event.3 \ man/ibv_get_cq_event.3 man/ibv_get_device_guid.3 \ man/ibv_get_device_list.3 man/ibv_get_device_name.3 \ - man/ibv_modify_qp.3 man/ibv_modify_srq.3 man/ibv_open_device.3 \ + man/ibv_modify_qp.3 man/ibv_modify_srq.3 man/ibv_modify_xrc_rcv_qp.3 \ + man/ibv_open_device.3 man/ibv_open_xrc_domain.3 \ man/ibv_poll_cq.3 man/ibv_post_recv.3 man/ibv_post_send.3 \ man/ibv_post_srq_recv.3 man/ibv_query_device.3 man/ibv_query_gid.3 \ man/ibv_query_pkey.3 man/ibv_query_port.3 man/ibv_query_qp.3 \ - man/ibv_query_srq.3 man/ibv_rate_to_mult.3 man/ibv_reg_mr.3 \ + man/ibv_query_srq.3 man/ibv_query_xrc_rcv_qp.3 \ + man/ibv_rate_to_mult.3 man/ibv_reg_mr.3 man/ibv_reg_xrc_rcv_qp.3 \ man/ibv_req_notify_cq.3 man/ibv_resize_cq.3 DEBIAN = debian/changelog debian/compat debian/control debian/copyright \ @@ -74,6 +77,8 @@ install-data-hook: $(RM) ibv_ack_async_event.3 && \ $(RM) ibv_ack_cq_events.3 && \ $(RM) ibv_close_device.3 && \ + $(RM) ibv_close_xrc_domain.3 && \ + $(RM) ibv_create_xrc_srq.3 && \ $(RM) ibv_dealloc_pd.3 && \ $(RM) ibv_dereg_mr.3 && \ $(RM) ibv_destroy_ah.3 && \ @@ -84,12 +89,15 @@ install-data-hook: $(RM) ibv_detach_mcast.3 && \ $(RM) ibv_free_device_list.3 && \ $(RM) ibv_init_ah_from_wc.3 && \ + $(RM) ibv_unreg_xrc_rcv_qp.3 && \ $(RM) mult_to_ibv_rate.3 && \ $(RM) ibv_node_type_str.3 && \ $(RM) ibv_port_state_str.3 && \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 && \ $(LN_S) ibv_get_cq_event.3 ibv_ack_cq_events.3 && \ $(LN_S) ibv_open_device.3 ibv_close_device.3 && \ + $(LN_S) ibv_open_xrc_domain.3 ibv_close_xrc_domain.3 && \ + $(LN_S) ibv_create_srq.3 ibv_create_xrc_srq.3 && \ $(LN_S) ibv_alloc_pd.3 ibv_dealloc_pd.3 && \ $(LN_S) ibv_reg_mr.3 ibv_dereg_mr.3 && \ $(LN_S) ibv_create_ah.3 ibv_destroy_ah.3 && \ @@ -100,6 +108,7 @@ install-data-hook: $(LN_S) ibv_attach_mcast.3 ibv_detach_mcast.3 && \ $(LN_S) ibv_get_device_list.3 ibv_free_device_list.3 && \ $(LN_S) ibv_create_ah_from_wc.3 ibv_init_ah_from_wc.3 && \ + $(LN_S) ibv_reg_xrc_rcv_qp.3 ibv_unreg_xrc_rcv_qp.3 && \ $(LN_S) ibv_rate_to_mult.3 mult_to_ibv_rate.3 && \ $(LN_S) ibv_event_type_str.3 ibv_node_type_str.3 && \ $(LN_S) ibv_event_type_str.3 ibv_port_state_str.3 diff --git a/man/ibv_create_qp.3 b/man/ibv_create_qp.3 index 28b3a09..106b31c 100644 --- a/man/ibv_create_qp.3 +++ b/man/ibv_create_qp.3 @@ -28,8 +28,9 @@ struct ibv_cq *send_cq; /* CQ to be associated with the Send Que struct ibv_cq *recv_cq; /* CQ to be associated with the Receive Queue (RQ) */ struct ibv_srq *srq; /* SRQ handle if QP is to be associated with an SRQ, otherwise NULL */ struct ibv_qp_cap cap; /* QP capabilities */ -enum ibv_qp_type qp_type; /* QP Transport Service Type: IBV_QPT_RC, IBV_QPT_UC, or IBV_QPT_UD */ +enum ibv_qp_type qp_type; /* QP Transport Service Type: IBV_QPT_RC, IBV_QPT_UC, IBV_QPT_UD or IBV_QPT_XRC */ int sq_sig_all; /* If set, each Work Request (WR) submitted to the SQ generates a completion entry */ +struct ibv_xrc_domain *xrc_domain; /* XRC domain the QP will be associated with (valid only for IBV_QPT_XRC QP), otherwise NULL */ .in -8 }; .sp diff --git a/man/ibv_create_srq.3 b/man/ibv_create_srq.3 index f0963e8..fae9c0c 100644 --- a/man/ibv_create_srq.3 +++ b/man/ibv_create_srq.3 @@ -10,12 +10,26 @@ ibv_create_srq, ibv_destroy_srq \- create or destroy a shared receive queue (SRQ .BI "struct ibv_srq *ibv_create_srq(struct ibv_pd " "*pd" ", struct " .BI " ibv_srq_init_attr " "*srq_init_attr" ); .sp +.BI "struct ibv_srq *ibv_create_xrc_srq(struct ibv_pd " "*pd" ", +.BI " struct ibv_xrc_domain " "*xrc_domain" ", +.BI " struct ibv_cq " "*xrc_cq" ", +.BI " struct ibv_srq_init_attr " "*srq_init_attr" ); +.sp .BI "int ibv_destroy_srq(struct ibv_srq " "*srq" ); .fi .SH "DESCRIPTION" .B ibv_create_srq() creates a shared receive queue (SRQ) associated with the protection domain .I pd\fR. +.PP +.B ibv_create_xrc_srq() +creates an XRC shared receive queue (SRQ) associated with the protection domain +.I pd\fR, +the XRC domain +.I xrc_domain +and the CQ which will hold the XRC completion +.I xrc_cq\fR. +.PP The argument .I srq_init_attr is an ibv_srq_init_attr struct, as defined in . diff --git a/man/ibv_create_xrc_rcv_qp.3 b/man/ibv_create_xrc_rcv_qp.3 new file mode 100644 index 0000000..8fc8119 --- /dev/null +++ b/man/ibv_create_xrc_rcv_qp.3 @@ -0,0 +1,70 @@ +.\" -*- nroff -*- +.\" +.TH IBV_CREATE_XRC_RCV_QP 3 2008-02-10 libibverbs "Libibverbs Programmer's Manual" +.SH "NAME" +ibv_create_xrc_rcv_qp \- create an XRC queue pair (QP) for serving as a receive-side only QP +.SH "SYNOPSIS" +.nf +.B #include +.sp +.BI "int ibv_create_xrc_rcv_qp(struct ibv_qp_init_attr " "*init_attr" , +.BI " uint32_t " "*xrc_rcv_qpn" ); +.fi +.SH "DESCRIPTION" +.B ibv_create_xrc_rcv_qp() +creates an XRC queue pair (QP) for serving as a receive-side only QP and returns its number through the pointer +.I xrc_rcv_qpn\fR. +This QP number should be passed to the remote node (sender). +The remote node will use +.I xrc_rcv_qpn +in +.B ibv_post_send() +when sending to an XRC SRQ on this host in the same xrc domain as the XRC receive QP. +This QP is created in kernel space, and persists until the last process registered for the QP +calls +.B ibv_unreg_xrc_rcv_qp() +(at which time the QP is destroyed). +.PP +The process which creates this QP is automatically registered for it, and should also call +.B ibv_unreg_xrc_rcv_qp() +at some point, to unregister. + +Processes which wish to receive on an XRC SRQ via this QP should call +.B ibv_reg_xrc_rcv_qp() +for this QP, to guarantee that the QP will not be destroyed while they are still using it for receiving on the XRC SRQ. +.PP +The argument +.I qp_init_attr +is an ibv_qp_init_attr struct, as defined in . +.PP +.nf +struct ibv_qp_init_attr { +.in +8 +void *qp_context; /* value is being ignored */ +struct ibv_cq *send_cq; /* value is being ignored */ +struct ibv_cq *recv_cq; /* value is being ignored */ +struct ibv_srq *srq; /* value is being ignored */ +struct ibv_qp_cap cap; /* value is being ignored */ +enum ibv_qp_type qp_type; /* value is being ignored */ +int sq_sig_all; /* value is being ignored */ +struct ibv_xrc_domain *xrc_domain; /* XRC domain the QP will be associated with */ +.in -8 +}; +.fi +.PP +Most of the attributes in +.I qp_init_attr +are being ignored because this QP is a receive only QP and all RR are being posted to an SRQ. +.SH "RETURN VALUE" +.B ibv_create_xrc_rcv_qp() +returns 0 on success, or the value of errno on failure (which indicates the failure reason). +.SH "SEE ALSO" +.BR ibv_open_xrc_domain (3), +.BR ibv_modify_xrc_rcv_qp (3), +.BR ibv_query_xrc_rcv_qp (3), +.BR ibv_reg_xrc_rcv_qp (3), +.BR ibv_unreg_xrc_rcv_qp (3), +.BR ibv_post_send (3) +.SH "AUTHORS" +.TP +Dotan Barak diff --git a/man/ibv_modify_xrc_rcv_qp.3 b/man/ibv_modify_xrc_rcv_qp.3 new file mode 100644 index 0000000..881eb3b --- /dev/null +++ b/man/ibv_modify_xrc_rcv_qp.3 @@ -0,0 +1,141 @@ +.\" -*- nroff -*- +.\" +.TH IBV_MODIFY_XRC_RCV_QP 3 2008-02-10 libibverbs "Libibverbs Programmer's Manual" +.SH "NAME" +ibv_modify_xrc_rcv_qp \- modify the attributes of an XRC receive queue pair (QP) +.SH "SYNOPSIS" +.nf +.B #include +.sp +.BI "int ibv_modify_xrc_rcv_qp(struct ibv_xrc_domain " "*xrc_domain" ", uint32_t " "xrc_qp_num" , +.BI " struct ibv_qp_attr " "*attr" ", int " "attr_mask" ); +.fi +.SH "DESCRIPTION" +.B ibv_modify_qp() +modifies the attributes of an XRC receive QP with the number +.I xrc_qp_num +which is associated with the XRC domain +.I xrc_domain +with the attributes in +.I attr +according to the mask +.I attr_mask +and move the QP state through the following transitions: Reset -> Init -> RTR. +.I attr_mask +should indicate all of the attributes which will be used in this QP transition and the following masks (at least) should be set: +.PP +.nf +Next state Required attributes +\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- +Init \fB IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, \fR + \fB IBV_QP_ACCESS_FLAGS \fR +RTR \fB IBV_QP_STATE, IBV_QP_AV, IBV_QP_PATH_MTU, \fR + \fB IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, \fR + \fB IBV_QP_MAX_DEST_RD_ATOMIC, IBV_QP_MIN_RNR_TIMER \fR +.fi +.PP +The user can add optional attributes as well. +.PP +The argument \fIattr\fR is an ibv_qp_attr struct, as defined in . +.PP +.nf +struct ibv_qp_attr { +.in +8 +enum ibv_qp_state qp_state; /* Move the QP to this state */ +enum ibv_qp_state cur_qp_state; /* Assume this is the current QP state */ +enum ibv_mtu path_mtu; /* Path MTU (valid only for RC/UC QPs) */ +enum ibv_mig_state path_mig_state; /* Path migration state (valid if HCA supports APM) */ +uint32_t qkey; /* Q_Key for the QP (valid only for UD QPs) */ +uint32_t rq_psn; /* PSN for receive queue (valid only for RC/UC QPs) */ +uint32_t sq_psn; /* PSN for send queue (valid only for RC/UC QPs) */ +uint32_t dest_qp_num; /* Destination QP number (valid only for RC/UC QPs) */ +int qp_access_flags; /* Mask of enabled remote access operations (valid only for RC/UC QPs) */ +struct ibv_qp_cap cap; /* QP capabilities (valid if HCA supports QP resizing) */ +struct ibv_ah_attr ah_attr; /* Primary path address vector (valid only for RC/UC QPs) */ +struct ibv_ah_attr alt_ah_attr; /* Alternate path address vector (valid only for RC/UC QPs) */ +uint16_t pkey_index; /* Primary P_Key index */ +uint16_t alt_pkey_index; /* Alternate P_Key index */ +uint8_t en_sqd_async_notify; /* Enable SQD.drained async notification (Valid only if qp_state is SQD) */ +uint8_t sq_draining; /* Is the QP draining? Irrelevant for ibv_modify_qp() */ +uint8_t max_rd_atomic; /* Number of outstanding RDMA reads & atomic operations on the destination QP (valid only for RC QPs) */ +uint8_t max_dest_rd_atomic; /* Number of responder resources for handling incoming RDMA reads & atomic operations (valid only for RC QPs) */ +uint8_t min_rnr_timer; /* Minimum RNR NAK timer (valid only for RC QPs) */ +uint8_t port_num; /* Primary port number */ +uint8_t timeout; /* Local ack timeout for primary path (valid only for RC QPs) */ +uint8_t retry_cnt; /* Retry count (valid only for RC QPs) */ +uint8_t rnr_retry; /* RNR retry (valid only for RC QPs) */ +uint8_t alt_port_num; /* Alternate port number */ +uint8_t alt_timeout; /* Local ack timeout for alternate path (valid only for RC QPs) */ +.in -8 +}; +.fi +.PP +For details on struct ibv_qp_cap see the description of +.B ibv_create_qp()\fR. +For details on struct ibv_ah_attr see the description of +.B ibv_create_ah()\fR. +.PP +The argument +.I attr_mask +specifies the QP attributes to be modified. +The argument is either 0 or the bitwise OR of one or more of the following flags: +.PP +.TP +.B IBV_QP_STATE \fR Modify qp_state +.TP +.B IBV_QP_CUR_STATE \fR Set cur_qp_state +.TP +.B IBV_QP_EN_SQD_ASYNC_NOTIFY \fR Set en_sqd_async_notify +.TP +.B IBV_QP_ACCESS_FLAGS \fR Set qp_access_flags +.TP +.B IBV_QP_PKEY_INDEX \fR Set pkey_index +.TP +.B IBV_QP_PORT \fR Set port_num +.TP +.B IBV_QP_QKEY \fR Set qkey +.TP +.B IBV_QP_AV \fR Set ah_attr +.TP +.B IBV_QP_PATH_MTU \fR Set path_mtu +.TP +.B IBV_QP_TIMEOUT \fR Set timeout +.TP +.B IBV_QP_RETRY_CNT \fR Set retry_cnt +.TP +.B IBV_QP_RNR_RETRY \fR Set rnr_retry +.TP +.B IBV_QP_RQ_PSN \fR Set rq_psn +.TP +.B IBV_QP_MAX_QP_RD_ATOMIC \fR Set max_rd_atomic +.TP +.B IBV_QP_ALT_PATH \fR Set the alternative path via: alt_ah_attr, alt_pkey_index, alt_port_num, alt_timeout +.TP +.B IBV_QP_MIN_RNR_TIMER \fR Set min_rnr_timer +.TP +.B IBV_QP_SQ_PSN \fR Set sq_psn +.TP +.B IBV_QP_MAX_DEST_RD_ATOMIC \fR Set max_dest_rd_atomic +.TP +.B IBV_QP_PATH_MIG_STATE \fR Set path_mig_state +.TP +.B IBV_QP_CAP \fR Set cap +.TP +.B IBV_QP_DEST_QPN \fR Set dest_qp_num +.SH "RETURN VALUE" +.B ibv_modify_xrc_rcv_qp() +returns 0 on success, or the value of errno on failure (which indicates the failure reason). +.SH "NOTES" +If any of the modify attributes or the modify mask are invalid, none +of the attributes will be modified (including the QP state). +.PP +Not all devices support alternate paths. To check if a device supports it, check if the +.B IBV_DEVICE_AUTO_PATH_MIG +bit is set in the device capabilities flags. +.SH "SEE ALSO" +.BR ibv_open_xrc_domain (3), +.BR ibv_create_xrc_rcv_qp (3), +.BR ibv_query_xrc_rcv_qp (3) +.SH "AUTHORS" +.TP +Dotan Barak diff --git a/man/ibv_open_xrc_domain.3 b/man/ibv_open_xrc_domain.3 new file mode 100644 index 0000000..f8032ea --- /dev/null +++ b/man/ibv_open_xrc_domain.3 @@ -0,0 +1,80 @@ +.\" -*- nroff -*- +.\" +.TH IBV_OPEN_XRC_DOMAIN 3 2008-02-10 libibverbs "Libibverbs Programmer's Manual" +.SH "NAME" +ibv_open_xrc_domain, ibv_close_xrc_domain \- open or close an eXtended Reliable Connection (XRC) domain +.SH "SYNOPSIS" +.nf +.B #include +.B #include +.sp +.BI "struct ibv_xrc_domain *ibv_open_xrc_domain(struct ibv_context " "*context" "," +.BI " int " "fd" ", int " "oflag" ); +.nl +.BI "int ibv_close_xrc_domain(struct ibv_xrc_domain " "*d" ); +.fi +.SH "DESCRIPTION" +.B ibv_open_xrc_domain() +open an XRC domain for the InfiniBand device context +.I context +or return a reference to an opened one\fR. +.I fd +is the file descriptor to be associated with the XRC domain. +The argument +.I oflag +describes the desired file creation attributes; it is either 0 or the bitwise OR of one or more of the following flags: +.PP +.TP +.B O_CREAT +If a domain belonging to device named by context is already associated with the inode, this flag has +no effect, except as noted under +.BR O_EXCL +below. Otherwise, a new XRC domain is created and is associated with inode specified by +.IR fd\fR. +.TP +.B O_EXCL +If +.BR O_EXCL +and +.BR O_CREAT +are set, open will fail if a domain associated with the inode exists. +The check for the existence of the domain and creation +of the domain if it does not exist is atomic with respect to other +processes executing open with +.IR fd +naming the same inode. +.PP +If +.I fd +equals -1, no inode is is associated with the domain, and the only valid value for +.I oflag +is +.B O_CREAT\fR. +.PP +.B ibv_close_xrc_domain() +closes the XRC domain +.I d\fR. +If this is the last reference, the XRC domain will be destroyed. +.SH "RETURN VALUE" +.B ibv_open_xrc_domain() +returns a pointer to an opened XRC, or NULL if the request fails. +.PP +.B ibv_close_xrc_domain() +returns 0 on success, or the value of errno on failure (which indicates the failure reason). +.SH "NOTES" +Not all devices support XRC. To check if a device supports it, check if the +.B IBV_DEVICE_XRC +bit is set in the device capabilities flags. +.PP +.B ibv_close_xrc_domain() +may fail if any QP or SRQ are still associated with the XRC domain being closed. +.SH "SEE ALSO" +.BR ibv_create_xrc_srq (3), +.BR ibv_create_qp (3), +.BR ibv_create_xrc_rcv_qp (3), +.BR ibv_modify_xrc_rcv_qp (3), +.BR ibv_query_xrc_rcv_qp (3), +.BR ibv_reg_xrc_rcv_qp (3) +.SH "AUTHORS" +.TP +Dotan Barak diff --git a/man/ibv_post_send.3 b/man/ibv_post_send.3 index 8c7b0eb..3b1d363 100644 --- a/man/ibv_post_send.3 +++ b/man/ibv_post_send.3 @@ -60,6 +60,7 @@ uint32_t remote_qkey; /* Q_Key number of the destination QP */ } ud; .in -8 } wr; +uint32_t xrc_remote_srq_num; /* SRQ number of the destination XRC */ .in -8 }; .sp @@ -76,15 +77,15 @@ uint32_t lkey; /* Key of the local Memory Regio Each QP Transport Service Type supports a specific set of opcodes, as shown in the following table: .PP .nf -OPCODE | IBV_QPT_UD | IBV_QPT_UC | IBV_QPT_RC -\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\- -IBV_WR_SEND | X | X | X -IBV_WR_SEND_WITH_IMM | X | X | X -IBV_WR_RDMA_WRITE | | X | X -IBV_WR_RDMA_WRITE_WITH_IMM | | X | X -IBV_WR_RDMA_READ | | | X -IBV_WR_ATOMIC_CMP_AND_SWP | | | X -IBV_WR_ATOMIC_FETCH_AND_ADD | | | X +OPCODE | IBV_QPT_UD | IBV_QPT_UC | IBV_QPT_RC | IBV_QPT_XRC +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\- +IBV_WR_SEND | X | X | X | X +IBV_WR_SEND_WITH_IMM | X | X | X | X +IBV_WR_RDMA_WRITE | | X | X | X +IBV_WR_RDMA_WRITE_WITH_IMM | | X | X | X +IBV_WR_RDMA_READ | | | X | X +IBV_WR_ATOMIC_CMP_AND_SWP | | | X | X +IBV_WR_ATOMIC_FETCH_AND_ADD | | | X | X .fi .PP The attribute send_flags describes the properties of the \s-1WR\s0. It is either 0 or the bitwise \s-1OR\s0 of one or more of the following flags: @@ -114,6 +115,7 @@ IBV_SEND_INLINE flag was set, the buffer can be reused immediately after the call returns. .SH "SEE ALSO" .BR ibv_create_qp (3), +.BR ibv_create_xrc_rcv_qp (3), .BR ibv_create_ah (3), .BR ibv_post_recv (3), .BR ibv_post_srq_recv (3), diff --git a/man/ibv_query_xrc_rcv_qp.3 b/man/ibv_query_xrc_rcv_qp.3 new file mode 100644 index 0000000..d2429be --- /dev/null +++ b/man/ibv_query_xrc_rcv_qp.3 @@ -0,0 +1,89 @@ +.\" -*- nroff -*- +.\" +.TH IBV_QUERY_XRC_RCV_QP 3 2008-02-10 libibverbs "Libibverbs Programmer's Manual" +.SH "NAME" +ibv_query_xrc_rcv_qp \- get the attributes of an XRC receive queue pair (QP) +.SH "SYNOPSIS" +.nf +.B #include +.sp +.BI "int ibv_query_xrc_rcv_qp(struct ibv_xrc_domain " "*xrc_domain" ", uint32_t " "xrc_qp_num" , +.BI " struct ibv_qp_attr " "*attr" ", int " "attr_mask" , +.BI " struct ibv_qp_init_attr " "*init_attr" ); +.fi +.SH "DESCRIPTION" +.B ibv_query_xrc_rcv_qp() +gets the attributes specified in +.I attr_mask +for the XRC receive QP with the number +.I xrc_qp_num +which is associated with the XRC domain +.I xrc_domain +and returns them through the pointers +.I attr +and +.I init_attr\fR. +The argument +.I attr +is an ibv_qp_attr struct, as defined in . +.PP +.nf +struct ibv_qp_attr { +.in +8 +enum ibv_qp_state qp_state; /* Current QP state */ +enum ibv_qp_state cur_qp_state; /* Current QP state - irrelevant for ibv_query_qp */ +enum ibv_mtu path_mtu; /* Path MTU (valid only for RC/UC QPs) */ +enum ibv_mig_state path_mig_state; /* Path migration state (valid if HCA supports APM) */ +uint32_t qkey; /* Q_Key of the QP (valid only for UD QPs) */ +uint32_t rq_psn; /* PSN for receive queue (valid only for RC/UC QPs) */ +uint32_t sq_psn; /* PSN for send queue (valid only for RC/UC QPs) */ +uint32_t dest_qp_num; /* Destination QP number (valid only for RC/UC QPs) */ +int qp_access_flags; /* Mask of enabled remote access operations (valid only for RC/UC QPs) */ +struct ibv_qp_cap cap; /* QP capabilities */ +struct ibv_ah_attr ah_attr; /* Primary path address vector (valid only for RC/UC QPs) */ +struct ibv_ah_attr alt_ah_attr; /* Alternate path address vector (valid only for RC/UC QPs) */ +uint16_t pkey_index; /* Primary P_Key index */ +uint16_t alt_pkey_index; /* Alternate P_Key index */ +uint8_t en_sqd_async_notify; /* Enable SQD.drained async notification - irrelevant for ibv_query_qp */ +uint8_t sq_draining; /* Is the QP draining? (Valid only if qp_state is SQD) */ +uint8_t max_rd_atomic; /* Number of outstanding RDMA reads & atomic operations on the destination QP (valid only for RC QPs) */ +uint8_t max_dest_rd_atomic; /* Number of responder resources for handling incoming RDMA reads & atomic operations (valid only for RC QPs) */ +uint8_t min_rnr_timer; /* Minimum RNR NAK timer (valid only for RC QPs) */ +uint8_t port_num; /* Primary port number */ +uint8_t timeout; /* Local ack timeout for primary path (valid only for RC QPs) */ +uint8_t retry_cnt; /* Retry count (valid only for RC QPs) */ +uint8_t rnr_retry; /* RNR retry (valid only for RC QPs) */ +uint8_t alt_port_num; /* Alternate port number */ +uint8_t alt_timeout; /* Local ack timeout for alternate path (valid only for RC QPs) */ +.in -8 +}; +.fi +.PP +For details on struct ibv_qp_cap see the description of +.B ibv_create_qp()\fR. +For details on struct ibv_ah_attr see the description of +.B ibv_create_ah()\fR. +.SH "RETURN VALUE" +.B ibv_query_xrc_rcv_qp() +returns 0 on success, or the value of errno on failure (which indicates the failure reason). +.SH "NOTES" +The argument +.I attr_mask +is a hint that specifies the minimum list of attributes to retrieve. +Some InfiniBand devices may return extra attributes not requested, for +example if the value can be returned cheaply. +.PP +Attribute values are valid if they have been set using +.B ibv_modify_xrc_rcv_qp()\fR. +The exact list of valid attributes depends on the QP state. +.PP +Multiple calls to +.B ibv_query_xrc_rcv_qp() +may yield some differences in the values returned for the following attributes: qp_state, path_mig_state, sq_draining, ah_attr (if APM is enabled). +.SH "SEE ALSO" +.BR ibv_open_xrc_domain (3), +.BR ibv_create_xrc_rcv_qp (3), +.BR ibv_modify_xrc_rcv_qp (3) +.SH "AUTHORS" +.TP +Dotan Barak diff --git a/man/ibv_reg_xrc_rcv_qp.3 b/man/ibv_reg_xrc_rcv_qp.3 new file mode 100644 index 0000000..767c9f3 --- /dev/null +++ b/man/ibv_reg_xrc_rcv_qp.3 @@ -0,0 +1,57 @@ +.\" -*- nroff -*- +.\" +.TH IBV_REG_XRC_RCV_QP 3 2008-10-02 libibverbs "Libibverbs Programmer's Manual" +.SH "NAME" +ibv_reg_xrc_rcv_qp, ibv_unreg_xrc_rcv_qp \- register and unregister a user process with an XRC receive queue pair (QP) +.SH "SYNOPSIS" +.nf +.B #include +.sp +.BI "int ibv_reg_xrc_rcv_qp(struct ibv_xrc_domain " "*xrc_domain" ", uint32_t " "xrc_qp_num" "); +.nl +.BI "int ibv_unreg_xrc_rcv_qp(struct ibv_xrc_domain " "*xrc_domain" ", uint32_t " "xrc_qp_num" "); +.fi +.SH "DESCRIPTION" +.B ibv_reg_xrc_rcv_qp() +registers a user process with the XRC receive QP (created via +.B ibv_create_xrc_rcv_qp() +) whose number is +.I xrc_qp_num\fR, +and which is associated with the XRC domain +.I xrc_domain\fR. +.PP +.B ibv_unreg_xrc_rcv_qp() +unregisters a user process from the XRC receive QP number +.I xrc_qp_num\fR, +which is associated with the XRC domain +.I xrc_domain\fR. +When the number of user processes registered with this XRC receive QP drops to zero, the QP is destroyed. +.SH "RETURN VALUE" +.B ibv_reg_xrc_rcv_qp() +and +.B ibv_unreg_xrc_rcv_qp() +returns 0 on success, or the value of errno on failure (which indicates the failure reason). +.SH "NOTES" +.B ibv_reg_xrc_rcv_qp() +and +.B ibv_unreg_xrc_rcv_qp() +may fail if the number +.I xrc_qp_num +is not a number of a valid XRC receive QP (the QP is not allocated or it is the number of a non-XRC QP), or +the XRC receive QP was created with an XRC domain other than +.I xrc_domain\fR. + +If a process is still registered with any XRC RCV QPs belonging to some domain, +.B ibv_close_xrc_domain() +will return failure if called for that domain in that process. + +.B ibv_create_xrc_rcv_qp() +performs an implicit registration for the creating process; when that process is finished with the XRC RCV QP, it should call +.B ibv_unreg_xrc_rcv_qp() +for that QP. Note that if no other processes are registered with the QP at this time, its registration count will drop to zero and it will be destroyed. +.SH "SEE ALSO" +.BR ibv_open_xrc_domain (3), +.BR ibv_create_xrc_rcv_qp (3) +.SH "AUTHORS" +.TP +Dotan Barak -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:15 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:15 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 1/12 V2] libibverbs: XRC base implementation Message-ID: <200807101851.15977.jackm@dev.mellanox.co.il> From aa4ad2fda6aecdbc0358bac9e28cf49fab886674 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Wed, 9 Jul 2008 12:46:43 +0300 Subject: [PATCH] Consolidated XRC implementation patch (from OFED 1.3 libibverbs). This patch includes the following commits from OFED 1.3 libibverbs: Implement eXtended Reliable Connections (a7df4af8eb84738f36db4161a4272fa02fc6741e) Re-define IBV_DEVICE_XRC to conform to its new position (5042a9cab0ae2f7ad61bdf88dfed6fb10b700797) Set "is_srq" flag only when the QP has an SRQ (6f6d29e74ca0c19a8821990aad603e3c575b7f4d) For XRC QPs, return xrc_domain in ibv_query_qp (018c44a44ff0344dfe7cf5f6598f81d81769164e) V2: 1. checkpatch.pl cleanups 2. fixed u64 alignment problems in uverbs.h ABI structs 3. eliminated unnecessary default_symvers 4. modified xrc_ops to more_ops Signed-off-by: Jack Morgenstein --- include/infiniband/driver.h | 11 +++++ include/infiniband/kern-abi.h | 47 ++++++++++++++++++++++- include/infiniband/verbs.h | 85 ++++++++++++++++++++++++++++++++++++++++- src/cmd.c | 72 ++++++++++++++++++++++++++++++++++- src/libibverbs.map | 6 +++ src/verbs.c | 54 ++++++++++++++++++++++++++ 6 files changed, 271 insertions(+), 4 deletions(-) diff --git a/include/infiniband/driver.h b/include/infiniband/driver.h index 67a3bf8..30ba79f 100644 --- a/include/infiniband/driver.h +++ b/include/infiniband/driver.h @@ -99,6 +99,11 @@ int ibv_cmd_create_srq(struct ibv_pd *pd, struct ibv_srq *srq, struct ibv_srq_init_attr *attr, struct ibv_create_srq *cmd, size_t cmd_size, struct ibv_create_srq_resp *resp, size_t resp_size); +int ibv_cmd_create_xrc_srq(struct ibv_pd *pd, + struct ibv_srq *srq, struct ibv_srq_init_attr *attr, + uint32_t xrc_domain, uint32_t xrc_cq, + struct ibv_create_xrc_srq *cmd, size_t cmd_size, + struct ibv_create_srq_resp *resp, size_t resp_size); int ibv_cmd_modify_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, enum ibv_srq_attr_mask srq_attr_mask, @@ -134,6 +139,12 @@ int ibv_cmd_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); int ibv_dontfork_range(void *base, size_t size); int ibv_dofork_range(void *base, size_t size); +int ibv_cmd_open_xrc_domain(struct ibv_context *context, int fd, int oflag, + struct ibv_xrc_domain *d, + struct ibv_open_xrc_domain_resp *resp, + size_t resp_size); +int ibv_cmd_close_xrc_domain(struct ibv_xrc_domain *d); + /* * sysfs helper functions diff --git a/include/infiniband/kern-abi.h b/include/infiniband/kern-abi.h index 0db083a..94cf3f2 100644 --- a/include/infiniband/kern-abi.h +++ b/include/infiniband/kern-abi.h @@ -85,7 +85,10 @@ enum { IB_USER_VERBS_CMD_MODIFY_SRQ, IB_USER_VERBS_CMD_QUERY_SRQ, IB_USER_VERBS_CMD_DESTROY_SRQ, - IB_USER_VERBS_CMD_POST_SRQ_RECV + IB_USER_VERBS_CMD_POST_SRQ_RECV, + IB_USER_VERBS_CMD_CREATE_XRC_SRQ, + IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN, + IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN }; /* @@ -706,6 +709,21 @@ struct ibv_create_srq { __u64 driver_data[0]; }; +struct ibv_create_xrc_srq { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u64 user_handle; + __u32 pd_handle; + __u32 max_wr; + __u32 max_sge; + __u32 srq_limit; + __u32 xrcd_handle; + __u32 xrc_cq; + __u64 driver_data[0]; +}; + struct ibv_create_srq_resp { __u32 srq_handle; __u32 max_wr; @@ -754,6 +772,30 @@ struct ibv_destroy_srq_resp { __u32 events_reported; }; +struct ibv_open_xrc_domain { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u32 fd; + __u32 oflags; + __u64 driver_data[0]; +}; + +struct ibv_open_xrc_domain_resp { + __u32 xrcd_handle; +}; + +struct ibv_close_xrc_domain { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u32 xrcd_handle; + __u32 reserved; + __u64 driver_data[0]; +}; + /* * Compatibility with older ABI versions */ @@ -803,6 +845,9 @@ enum { * trick opcodes in IBV_INIT_CMD() doesn't break. */ IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL_V2 = -1, + IB_USER_VERBS_CMD_CREATE_XRC_SRQ_V2 = -1, + IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN_V2 = -1, + IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN_V2 = -1, }; struct ibv_destroy_cq_v1 { diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index a04cc62..6c9a3b3 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -92,7 +92,8 @@ enum ibv_device_cap_flags { IBV_DEVICE_SYS_IMAGE_GUID = 1 << 11, IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, IBV_DEVICE_SRQ_RESIZE = 1 << 13, - IBV_DEVICE_N_NOTIFY_CQ = 1 << 14 + IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, + IBV_DEVICE_XRC = 1 << 20 }; enum ibv_atomic_cap { @@ -371,6 +372,11 @@ struct ibv_ah_attr { uint8_t port_num; }; +struct ibv_xrc_domain { + struct ibv_context *context; + uint32_t handle; +}; + enum ibv_srq_attr_mask { IBV_SRQ_MAX_WR = 1 << 0, IBV_SRQ_LIMIT = 1 << 1 @@ -390,7 +396,8 @@ struct ibv_srq_init_attr { enum ibv_qp_type { IBV_QPT_RC = 2, IBV_QPT_UC, - IBV_QPT_UD + IBV_QPT_UD, + IBV_QPT_XRC }; struct ibv_qp_cap { @@ -409,6 +416,7 @@ struct ibv_qp_init_attr { struct ibv_qp_cap cap; enum ibv_qp_type qp_type; int sq_sig_all; + struct ibv_xrc_domain *xrc_domain; }; enum ibv_qp_attr_mask { @@ -527,6 +535,7 @@ struct ibv_send_wr { uint32_t remote_qkey; } ud; } wr; + uint32_t xrc_remote_srq_num; }; struct ibv_recv_wr { @@ -554,6 +563,10 @@ struct ibv_srq { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + uint32_t xrc_srq_num; + struct ibv_xrc_domain *xrc_domain; + struct ibv_cq *xrc_cq; }; struct ibv_qp { @@ -571,6 +584,8 @@ struct ibv_qp { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + struct ibv_xrc_domain *xrc_domain; }; struct ibv_comp_channel { @@ -625,6 +640,16 @@ struct ibv_device { char ibdev_path[IBV_SYSFS_PATH_MAX]; }; +struct ibv_more_ops { + struct ibv_srq * (*create_xrc_srq)(struct ibv_pd *pd, + struct ibv_xrc_domain *xrc_domain, + struct ibv_cq *xrc_cq, + struct ibv_srq_init_attr *srq_init_attr); + struct ibv_xrc_domain * (*open_xrc_domain)(struct ibv_context *context, + int fd, int oflag); + int (*close_xrc_domain)(struct ibv_xrc_domain *d); +}; + struct ibv_context_ops { int (*query_device)(struct ibv_context *context, struct ibv_device_attr *device_attr); @@ -691,6 +716,7 @@ struct ibv_context { int num_comp_vectors; pthread_mutex_t mutex; void *abi_compat; + struct ibv_more_ops *more_ops; }; /** @@ -913,6 +939,25 @@ struct ibv_srq *ibv_create_srq(struct ibv_pd *pd, struct ibv_srq_init_attr *srq_init_attr); /** + * ibv_create_xrc_srq - Creates a SRQ associated with the specified protection + * domain and xrc domain. + * @pd: The protection domain associated with the SRQ. + * @xrc_domain: The XRC domain associated with the SRQ. + * @xrc_cq: CQ to report completions for XRC packets on. + * + * @srq_init_attr: A list of initial attributes required to create the SRQ. + * + * srq_attr->max_wr and srq_attr->max_sge are read the determine the + * requested size of the SRQ, and set to the actual values allocated + * on return. If ibv_create_srq() succeeds, then max_wr and max_sge + * will always be at least as large as the requested values. + */ +struct ibv_srq *ibv_create_xrc_srq(struct ibv_pd *pd, + struct ibv_xrc_domain *xrc_domain, + struct ibv_cq *xrc_cq, + struct ibv_srq_init_attr *srq_init_attr); + +/** * ibv_modify_srq - Modifies the attributes for the specified SRQ. * @srq: The SRQ to modify. * @srq_attr: On input, specifies the SRQ attributes to modify. On output, @@ -1093,6 +1138,42 @@ const char *ibv_port_state_str(enum ibv_port_state port_state); */ const char *ibv_event_type_str(enum ibv_event_type event); +/** + * ibv_open_xrc_domain - open an XRC domain + * Returns a reference to an XRC domain. + * + * @context: Device context + * @fd: descriptor for inode associated with the domain + * If fd == -1, no inode is associated with the domain; in this case, + * the only legal value for oflag is O_CREAT + * + * @oflag: oflag values are constructed by OR-ing flags from the following list + * + * O_CREAT + * If a domain belonging to device named by context is already associated + * with the inode, this flag has no effect, except as noted under O_EXCL + * below. Otherwise, a new XRC domain is created and is associated with + * inode specified by fd. + * + * O_EXCL + * If O_EXCL and O_CREAT are set, open will fail if a domain associated with + * the inode exists. The check for the existence of the domain and creation + * of the domain if it does not exist is atomic with respect to other + * processes executing open with fd naming the same inode. + */ +struct ibv_xrc_domain *ibv_open_xrc_domain(struct ibv_context *context, + int fd, int oflag); + +/** + * ibv_close_xrc_domain - close an XRC domain + * If this is the last reference, destroys the domain. + * + * @d: reference to XRC domain to close + * + * close is implicitly performed at process exit. + */ +int ibv_close_xrc_domain(struct ibv_xrc_domain *d); + END_C_DECLS # undef __attribute_const diff --git a/src/cmd.c b/src/cmd.c index 66d7134..66e3f2d 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -483,6 +483,34 @@ int ibv_cmd_create_srq(struct ibv_pd *pd, return 0; } +int ibv_cmd_create_xrc_srq(struct ibv_pd *pd, + struct ibv_srq *srq, struct ibv_srq_init_attr *attr, + uint32_t xrcd_handle, uint32_t xrc_cq, + struct ibv_create_xrc_srq *cmd, size_t cmd_size, + struct ibv_create_srq_resp *resp, size_t resp_size) +{ + IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_XRC_SRQ, resp, resp_size); + cmd->user_handle = (uintptr_t) srq; + cmd->pd_handle = pd->handle; + cmd->max_wr = attr->attr.max_wr; + cmd->max_sge = attr->attr.max_sge; + cmd->srq_limit = attr->attr.srq_limit; + cmd->xrcd_handle = xrcd_handle; + cmd->xrc_cq = xrc_cq; + + if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) + return errno; + + VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); + + srq->handle = resp->srq_handle; + srq->context = pd->context; + attr->attr.max_wr = resp->max_wr; + attr->attr.max_sge = resp->max_sge; + + return 0; +} + static int ibv_cmd_modify_srq_v3(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, enum ibv_srq_attr_mask srq_attr_mask, @@ -603,7 +631,6 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, cmd->pd_handle = pd->handle; cmd->send_cq_handle = attr->send_cq->handle; cmd->recv_cq_handle = attr->recv_cq->handle; - cmd->srq_handle = attr->srq ? attr->srq->handle : 0; cmd->max_send_wr = attr->cap.max_send_wr; cmd->max_recv_wr = attr->cap.max_recv_wr; cmd->max_send_sge = attr->cap.max_send_sge; @@ -612,6 +639,9 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, cmd->sq_sig_all = attr->sq_sig_all; cmd->qp_type = attr->qp_type; cmd->is_srq = !!attr->srq; + cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? + (attr->xrc_domain ? attr->xrc_domain->handle : 0) : + (attr->srq ? attr->srq->handle : 0); cmd->reserved = 0; if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) @@ -722,6 +752,8 @@ int ibv_cmd_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, init_attr->recv_cq = qp->recv_cq; init_attr->srq = qp->srq; init_attr->qp_type = qp->qp_type; + if (qp->qp_type == IBV_QPT_XRC) + init_attr->xrc_domain = qp->xrc_domain; init_attr->cap.max_send_wr = resp.max_send_wr; init_attr->cap.max_recv_wr = resp.max_recv_wr; init_attr->cap.max_send_sge = resp.max_send_sge; @@ -1122,3 +1154,41 @@ int ibv_cmd_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) return 0; } + +int ibv_cmd_open_xrc_domain(struct ibv_context *context, int fd, int oflag, + struct ibv_xrc_domain *d, + struct ibv_open_xrc_domain_resp *resp, + size_t resp_size) +{ + struct ibv_open_xrc_domain cmd; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD_RESP(&cmd, sizeof cmd, OPEN_XRC_DOMAIN, resp, resp_size); + cmd.fd = fd; + cmd.oflags = oflag; + + if (write(context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) + return errno; + + d->handle = resp->xrcd_handle; + + return 0; +} + +int ibv_cmd_close_xrc_domain(struct ibv_xrc_domain *d) +{ + struct ibv_close_xrc_domain cmd; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD(&cmd, sizeof cmd, CLOSE_XRC_DOMAIN); + cmd.xrcd_handle = d->handle; + + if (write(d->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) + return errno; + return 0; +} + diff --git a/src/libibverbs.map b/src/libibverbs.map index 1827da0..d1a6a47 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -91,6 +91,12 @@ IBVERBS_1.1 { ibv_dontfork_range; ibv_dofork_range; ibv_register_driver; + ibv_create_xrc_srq; + ibv_cmd_create_xrc_srq; + ibv_open_xrc_domain; + ibv_cmd_open_xrc_domain; + ibv_close_xrc_domain; + ibv_cmd_close_xrc_domain; ibv_node_type_str; ibv_port_state_str; diff --git a/src/verbs.c b/src/verbs.c index 9e370ce..f09542f 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -366,6 +366,9 @@ struct ibv_srq *__ibv_create_srq(struct ibv_pd *pd, srq->context = pd->context; srq->srq_context = srq_init_attr->srq_context; srq->pd = pd; + srq->xrc_domain = NULL; + srq->xrc_cq = NULL; + srq->xrc_srq_num = 0; srq->events_completed = 0; pthread_mutex_init(&srq->mutex, NULL); pthread_cond_init(&srq->cond, NULL); @@ -375,6 +378,32 @@ struct ibv_srq *__ibv_create_srq(struct ibv_pd *pd, } default_symver(__ibv_create_srq, ibv_create_srq); +struct ibv_srq *ibv_create_xrc_srq(struct ibv_pd *pd, + struct ibv_xrc_domain *xrc_domain, + struct ibv_cq *xrc_cq, + struct ibv_srq_init_attr *srq_init_attr) +{ + struct ibv_srq *srq; + + if (!pd->context->more_ops) + return NULL; + + srq = pd->context->more_ops->create_xrc_srq(pd, xrc_domain, + xrc_cq, srq_init_attr); + if (srq) { + srq->context = pd->context; + srq->srq_context = srq_init_attr->srq_context; + srq->pd = pd; + srq->xrc_domain = xrc_domain; + srq->xrc_cq = xrc_cq; + srq->events_completed = 0; + pthread_mutex_init(&srq->mutex, NULL); + pthread_cond_init(&srq->cond, NULL); + } + + return srq; +} + int __ibv_modify_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, enum ibv_srq_attr_mask srq_attr_mask) @@ -410,6 +439,8 @@ struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, qp->qp_type = qp_init_attr->qp_type; qp->state = IBV_QPS_RESET; qp->events_completed = 0; + qp->xrc_domain = qp_init_attr->qp_type == IBV_QPT_XRC ? + qp_init_attr->xrc_domain : NULL; pthread_mutex_init(&qp->mutex, NULL); pthread_cond_init(&qp->cond, NULL); } @@ -543,3 +574,26 @@ int __ibv_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) return qp->context->ops.detach_mcast(qp, gid, lid); } default_symver(__ibv_detach_mcast, ibv_detach_mcast); + +struct ibv_xrc_domain *ibv_open_xrc_domain(struct ibv_context *context, + int fd, int oflag) +{ + struct ibv_xrc_domain *d; + + if (!context->more_ops) + return NULL; + + d = context->more_ops->open_xrc_domain(context, fd, oflag); + if (d) + d->context = context; + + return d; +} + +int ibv_close_xrc_domain(struct ibv_xrc_domain *d) +{ + if (!d->context->more_ops) + return 0; + + return d->context->more_ops->close_xrc_domain(d); +} -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:22 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:22 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 2/12 V2] libibverbs: implement XRC RCV qps Message-ID: <200807101851.22914.jackm@dev.mellanox.co.il> From c33e87deccc1c59278e259c59ac82e3e548b0981 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Wed, 9 Jul 2008 13:01:31 +0300 Subject: [PATCH] Added support for XRC receive-only QPs. (OFED 1.3 libibverbs commit 6e99cddf835d4715ea7ca3641944e6285f27f2df) V2: 1. checkpatch.pl cleanups 2. Fixed u64 alignment problems in kern-abi.h 3. eliminated unneeded default_symvers 4. Added ibv_xrc_rcv_xxx lines to libibverbs.map IBVERBS_1.1 Signed-off-by: Jack Morgenstein --- include/infiniband/driver.h | 12 ++- include/infiniband/kern-abi.h | 99 +++++++++++++++++++- include/infiniband/verbs.h | 123 +++++++++++++++++++++++ src/cmd.c | 215 +++++++++++++++++++++++++++++++++++++++++ src/device.c | 52 +++++----- src/libibverbs.map | 10 ++ src/verbs.c | 59 +++++++++++ 7 files changed, 543 insertions(+), 27 deletions(-) diff --git a/include/infiniband/driver.h b/include/infiniband/driver.h index 30ba79f..f8138ef 100644 --- a/include/infiniband/driver.h +++ b/include/infiniband/driver.h @@ -144,7 +144,17 @@ int ibv_cmd_open_xrc_domain(struct ibv_context *context, int fd, int oflag, struct ibv_open_xrc_domain_resp *resp, size_t resp_size); int ibv_cmd_close_xrc_domain(struct ibv_xrc_domain *d); - +int ibv_cmd_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_rcv_qpn); +int ibv_cmd_modify_xrc_rcv_qp(struct ibv_xrc_domain *d, uint32_t xrc_rcv_qpn, + struct ibv_qp_attr *attr, int attr_mask); +int ibv_cmd_query_xrc_rcv_qp(struct ibv_xrc_domain *d, uint32_t xrc_rcv_qpn, + struct ibv_qp_attr *attr, int attr_mask, + struct ibv_qp_init_attr *init_attr); +int ibv_cmd_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); +int ibv_cmd_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); /* * sysfs helper functions diff --git a/include/infiniband/kern-abi.h b/include/infiniband/kern-abi.h index 94cf3f2..8b5cd9a 100644 --- a/include/infiniband/kern-abi.h +++ b/include/infiniband/kern-abi.h @@ -88,7 +88,12 @@ enum { IB_USER_VERBS_CMD_POST_SRQ_RECV, IB_USER_VERBS_CMD_CREATE_XRC_SRQ, IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN, - IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN + IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN, + IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP, + IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP, + IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, + IB_USER_VERBS_CMD_REG_XRC_RCV_QP, + IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, }; /* @@ -570,6 +575,93 @@ struct ibv_destroy_qp_resp { __u32 events_reported; }; +struct ibv_create_xrc_rcv_qp { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u64 user_handle; + __u32 xrc_domain_handle; + __u32 max_send_wr; + __u32 max_recv_wr; + __u32 max_send_sge; + __u32 max_recv_sge; + __u32 max_inline_data; + __u8 sq_sig_all; + __u8 qp_type; + __u8 reserved[6]; + __u64 driver_data[0]; +}; + +struct ibv_create_xrc_rcv_qp_resp { + __u32 qpn; + __u32 reserved; +}; + +struct ibv_modify_xrc_rcv_qp { + __u32 command; + __u16 in_words; + __u16 out_words; + __u32 xrc_domain_handle; + __u32 qp_num; + struct ibv_qp_dest dest; + struct ibv_qp_dest alt_dest; + __u32 attr_mask; + __u32 qkey; + __u32 rq_psn; + __u32 sq_psn; + __u32 dest_qp_num; + __u32 qp_access_flags; + __u16 pkey_index; + __u16 alt_pkey_index; + __u8 qp_state; + __u8 cur_qp_state; + __u8 path_mtu; + __u8 path_mig_state; + __u8 en_sqd_async_notify; + __u8 max_rd_atomic; + __u8 max_dest_rd_atomic; + __u8 min_rnr_timer; + __u8 port_num; + __u8 timeout; + __u8 retry_cnt; + __u8 rnr_retry; + __u8 alt_port_num; + __u8 alt_timeout; + __u8 reserved[6]; + __u64 driver_data[0]; +}; + +struct ibv_query_xrc_rcv_qp { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u32 xrc_domain_handle; + __u32 qp_num; + __u32 attr_mask; + __u32 reserved; + __u64 driver_data[0]; +}; + +struct ibv_reg_xrc_rcv_qp { + __u32 command; + __u16 in_words; + __u16 out_words; + __u32 xrc_domain_handle; + __u32 qp_num; + __u64 driver_data[0]; +}; + +struct ibv_unreg_xrc_rcv_qp { + __u32 command; + __u16 in_words; + __u16 out_words; + __u32 xrc_domain_handle; + __u32 qp_num; + __u64 driver_data[0]; +}; + struct ibv_kern_send_wr { __u64 wr_id; __u32 num_sge; @@ -848,6 +940,11 @@ enum { IB_USER_VERBS_CMD_CREATE_XRC_SRQ_V2 = -1, IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN_V2 = -1, IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN_V2 = -1, + IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_REG_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP_V2 = -1, }; struct ibv_destroy_cq_v1 { diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index 6c9a3b3..036a0c5 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -205,12 +205,17 @@ enum ibv_event_type { IBV_EVENT_CLIENT_REREGISTER }; +enum ibv_event_flags { + IBV_XRC_QP_EVENT_FLAG = 0x80000000, +}; + struct ibv_async_event { union { struct ibv_cq *cq; struct ibv_qp *qp; struct ibv_srq *srq; int port_num; + uint32_t xrc_qp_num; } element; enum ibv_event_type event_type; }; @@ -648,6 +653,22 @@ struct ibv_more_ops { struct ibv_xrc_domain * (*open_xrc_domain)(struct ibv_context *context, int fd, int oflag); int (*close_xrc_domain)(struct ibv_xrc_domain *d); + int (*create_xrc_rcv_qp)(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_qp_num); + int (*modify_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, + int attr_mask); + int (*query_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, + int attr_mask, + struct ibv_qp_init_attr *init_attr); + int (*reg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); + int (*unreg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); + }; struct ibv_context_ops { @@ -1174,6 +1195,108 @@ struct ibv_xrc_domain *ibv_open_xrc_domain(struct ibv_context *context, */ int ibv_close_xrc_domain(struct ibv_xrc_domain *d); +/** + * ibv_create_xrc_rcv_qp - creates an XRC QP for serving as a receive-side only QP, + * + * This QP is created in kernel space, and persists until the last process + * registered for the QP calls ibv_unreg_xrc_rcv_qp() (at which time the QP + * is destroyed). + * + * @init_attr: init attributes to use for QP. xrc domain MUST be included here. + * All other fields are ignored. + * + * @xrc_rcv_qpn: qp_num of created QP (if success). To be passed to the + * remote node (sender). The remote node will use xrc_rcv_qpn + * in ibv_post_send when sending to XRC SRQ's on this host + * in the same xrc domain. + * + * RETURNS: success (0), or a (negative) error value. + * + * NOTE: this verb also registers the calling user-process with the QP at its + * creation time (implicit call to ibv_reg_xrc_rcv_qp), to avoid race + * conditions. The creating process will need to call ibv_unreg_xrc_qp() + * for the QP to release it from this process. + */ +int ibv_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_rcv_qpn); + +/** + * ibv_modify_xrc_rcv_qp - modifies an xrc_rcv qp. + * + * @xrc_domain: xrc domain the QP belongs to (for verification). + * @xrc_qp_num: The (24 bit) number of the XRC QP. + * @attr: modify-qp attributes. The following fields must be specified: + * for RESET_2_INIT: qp_state, pkey_index , port, qp_access_flags + * for INIT_2_RTR: qp_state, path_mtu, dest_qp_num, rq_psn, + * max_dest_rd_atomic, min_rnr_timer, ah_attr + * The QP need not be brought to RTS for the QP to operate as a + * receive-only QP. + * @attr_mask: bitmap indicating which attributes are provided in the attr + * struct. Used for validity checking. + * The following bits must be set: + * for RESET_2_INIT: IBV_QP_PKEY_INDEX, IBV_QP_PORT, + * IBV_QP_ACCESS_FLAGS, IBV_QP_STATE + * for INIT_2_RTR: IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, + * IBV_QP_RQ_PSN, IBV_QP_MAX_DEST_RD_ATOMIC, + * IBV_QP_MIN_RNR_TIMER, IBV_QP_STATE + * + * RETURNS: success (0), or a (positive) error value. + * + */ +int ibv_modify_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, int attr_mask); + +/** + * ibv_query_xrc_rcv_qp - queries an xrc_rcv qp. + * + * @xrc_domain: xrc domain the QP belongs to (for verification). + * @xrc_qp_num: The (24 bit) number of the XRC QP. + * @attr: for returning qp attributes. + * @attr_mask: bitmap indicating which attributes to return. + * @init_attr: for returning the init attributes + * + * RETURNS: success (0), or a (positive) error value. + * + */ +int ibv_query_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, int attr_mask, + struct ibv_qp_init_attr *init_attr); + +/** + * ibv_reg_xrc_rcv_qp: registers a user process with an XRC QP which serves as + * a receive-side only QP. + * + * @xrc_domain: xrc domain the QP belongs to (for verification). + * @xrc_qp_num: The (24 bit) number of the XRC QP. + * + * RETURNS: success (0), + * or error (EINVAL), if: + * 1. There is no such QP_num allocated. + * 2. The QP is allocated, but is not an receive XRC QP + * 3. The XRC QP does not belong to the given domain. + */ +int ibv_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); + +/** + * ibv_unreg_xrc_rcv_qp: detaches a user process from an XRC QP serving as + * a receive-side only QP. If as a result, there are no remaining + * userspace processes registered for this XRC QP, it is destroyed. + * + * @xrc_domain: xrc domain the QP belongs to (for verification). + * @xrc_qp_num: The (24 bit) number of the XRC QP. + * + * RETURNS: success (0), + * or error (EINVAL), if: + * 1. There is no such QP_num allocated. + * 2. The QP is allocated, but is not an XRC QP + * 3. The XRC QP does not belong to the given domain. + * NOTE: There is no reason to return a special code if the QP is destroyed. + * The unregister simply succeeds. + */ +int ibv_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, + uint32_t xrc_qp_num); + END_C_DECLS # undef __attribute_const diff --git a/src/cmd.c b/src/cmd.c index 66e3f2d..cedf55e 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -828,6 +828,188 @@ int ibv_cmd_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, return 0; } +int ibv_cmd_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_rcv_qpn) +{ + struct ibv_create_xrc_rcv_qp cmd; + struct ibv_create_xrc_rcv_qp_resp resp; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD_RESP(&cmd, sizeof cmd, CREATE_XRC_RCV_QP, &resp, + sizeof resp); + + cmd.xrc_domain_handle = init_attr->xrc_domain->handle; + cmd.max_send_wr = init_attr->cap.max_send_wr; + cmd.max_recv_wr = init_attr->cap.max_recv_wr; + cmd.max_send_sge = init_attr->cap.max_send_sge; + cmd.max_recv_sge = init_attr->cap.max_recv_sge; + cmd.max_inline_data = init_attr->cap.max_inline_data; + cmd.sq_sig_all = init_attr->sq_sig_all; + cmd.qp_type = init_attr->qp_type; + cmd.reserved[0] = cmd.reserved[1] = 0; + + if (write(init_attr->xrc_domain->context->cmd_fd, &cmd, sizeof cmd) != + sizeof cmd) + return errno; + + *xrc_rcv_qpn = resp.qpn; + + return 0; +} + +int ibv_cmd_modify_xrc_rcv_qp(struct ibv_xrc_domain *d, uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, int attr_mask) +{ + struct ibv_modify_xrc_rcv_qp cmd; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD(&cmd, sizeof cmd, MODIFY_XRC_RCV_QP); + + cmd.xrc_domain_handle = d->handle; + cmd.qp_num = xrc_qp_num; + cmd.attr_mask = attr_mask; + cmd.qkey = attr->qkey; + cmd.rq_psn = attr->rq_psn; + cmd.sq_psn = attr->sq_psn; + cmd.dest_qp_num = attr->dest_qp_num; + cmd.qp_access_flags = attr->qp_access_flags; + cmd.pkey_index = attr->pkey_index; + cmd.alt_pkey_index = attr->alt_pkey_index; + cmd.qp_state = attr->qp_state; + cmd.cur_qp_state = attr->cur_qp_state; + cmd.path_mtu = attr->path_mtu; + cmd.path_mig_state = attr->path_mig_state; + cmd.en_sqd_async_notify = attr->en_sqd_async_notify; + cmd.max_rd_atomic = attr->max_rd_atomic; + cmd.max_dest_rd_atomic = attr->max_dest_rd_atomic; + cmd.min_rnr_timer = attr->min_rnr_timer; + cmd.port_num = attr->port_num; + cmd.timeout = attr->timeout; + cmd.retry_cnt = attr->retry_cnt; + cmd.rnr_retry = attr->rnr_retry; + cmd.alt_port_num = attr->alt_port_num; + cmd.alt_timeout = attr->alt_timeout; + + memcpy(cmd.dest.dgid, attr->ah_attr.grh.dgid.raw, 16); + cmd.dest.flow_label = attr->ah_attr.grh.flow_label; + cmd.dest.dlid = attr->ah_attr.dlid; + cmd.dest.reserved = 0; + cmd.dest.sgid_index = attr->ah_attr.grh.sgid_index; + cmd.dest.hop_limit = attr->ah_attr.grh.hop_limit; + cmd.dest.traffic_class = attr->ah_attr.grh.traffic_class; + cmd.dest.sl = attr->ah_attr.sl; + cmd.dest.src_path_bits = attr->ah_attr.src_path_bits; + cmd.dest.static_rate = attr->ah_attr.static_rate; + cmd.dest.is_global = attr->ah_attr.is_global; + cmd.dest.port_num = attr->ah_attr.port_num; + + memcpy(cmd.alt_dest.dgid, attr->alt_ah_attr.grh.dgid.raw, 16); + cmd.alt_dest.flow_label = attr->alt_ah_attr.grh.flow_label; + cmd.alt_dest.dlid = attr->alt_ah_attr.dlid; + cmd.alt_dest.reserved = 0; + cmd.alt_dest.sgid_index = attr->alt_ah_attr.grh.sgid_index; + cmd.alt_dest.hop_limit = attr->alt_ah_attr.grh.hop_limit; + cmd.alt_dest.traffic_class = attr->alt_ah_attr.grh.traffic_class; + cmd.alt_dest.sl = attr->alt_ah_attr.sl; + cmd.alt_dest.src_path_bits = attr->alt_ah_attr.src_path_bits; + cmd.alt_dest.static_rate = attr->alt_ah_attr.static_rate; + cmd.alt_dest.is_global = attr->alt_ah_attr.is_global; + cmd.alt_dest.port_num = attr->alt_ah_attr.port_num; + + cmd.reserved[0] = cmd.reserved[1] = 0; + + if (write(d->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) + return errno; + + return 0; +} + +int ibv_cmd_query_xrc_rcv_qp(struct ibv_xrc_domain *d, uint32_t xrc_qp_num, + struct ibv_qp_attr *attr, int attr_mask, + struct ibv_qp_init_attr *init_attr) +{ + struct ibv_query_xrc_rcv_qp cmd; + struct ibv_query_qp_resp resp; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD_RESP(&cmd, sizeof cmd, QUERY_XRC_RCV_QP, &resp, + sizeof resp); + cmd.xrc_domain_handle = d->handle; + cmd.qp_num = xrc_qp_num; + cmd.attr_mask = attr_mask; + + if (write(d->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) + return errno; + + VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + + attr->qkey = resp.qkey; + attr->rq_psn = resp.rq_psn; + attr->sq_psn = resp.sq_psn; + attr->dest_qp_num = resp.dest_qp_num; + attr->qp_access_flags = resp.qp_access_flags; + attr->pkey_index = resp.pkey_index; + attr->alt_pkey_index = resp.alt_pkey_index; + attr->qp_state = resp.qp_state; + attr->cur_qp_state = resp.cur_qp_state; + attr->path_mtu = resp.path_mtu; + attr->path_mig_state = resp.path_mig_state; + attr->sq_draining = resp.sq_draining; + attr->max_rd_atomic = resp.max_rd_atomic; + attr->max_dest_rd_atomic = resp.max_dest_rd_atomic; + attr->min_rnr_timer = resp.min_rnr_timer; + attr->port_num = resp.port_num; + attr->timeout = resp.timeout; + attr->retry_cnt = resp.retry_cnt; + attr->rnr_retry = resp.rnr_retry; + attr->alt_port_num = resp.alt_port_num; + attr->alt_timeout = resp.alt_timeout; + attr->cap.max_send_wr = resp.max_send_wr; + attr->cap.max_recv_wr = resp.max_recv_wr; + attr->cap.max_send_sge = resp.max_send_sge; + attr->cap.max_recv_sge = resp.max_recv_sge; + attr->cap.max_inline_data = resp.max_inline_data; + + memcpy(attr->ah_attr.grh.dgid.raw, resp.dest.dgid, 16); + attr->ah_attr.grh.flow_label = resp.dest.flow_label; + attr->ah_attr.dlid = resp.dest.dlid; + attr->ah_attr.grh.sgid_index = resp.dest.sgid_index; + attr->ah_attr.grh.hop_limit = resp.dest.hop_limit; + attr->ah_attr.grh.traffic_class = resp.dest.traffic_class; + attr->ah_attr.sl = resp.dest.sl; + attr->ah_attr.src_path_bits = resp.dest.src_path_bits; + attr->ah_attr.static_rate = resp.dest.static_rate; + attr->ah_attr.is_global = resp.dest.is_global; + attr->ah_attr.port_num = resp.dest.port_num; + + memcpy(attr->alt_ah_attr.grh.dgid.raw, resp.alt_dest.dgid, 16); + attr->alt_ah_attr.grh.flow_label = resp.alt_dest.flow_label; + attr->alt_ah_attr.dlid = resp.alt_dest.dlid; + attr->alt_ah_attr.grh.sgid_index = resp.alt_dest.sgid_index; + attr->alt_ah_attr.grh.hop_limit = resp.alt_dest.hop_limit; + attr->alt_ah_attr.grh.traffic_class = resp.alt_dest.traffic_class; + attr->alt_ah_attr.sl = resp.alt_dest.sl; + attr->alt_ah_attr.src_path_bits = resp.alt_dest.src_path_bits; + attr->alt_ah_attr.static_rate = resp.alt_dest.static_rate; + attr->alt_ah_attr.is_global = resp.alt_dest.is_global; + attr->alt_ah_attr.port_num = resp.alt_dest.port_num; + + init_attr->cap.max_send_wr = resp.max_send_wr; + init_attr->cap.max_recv_wr = resp.max_recv_wr; + init_attr->cap.max_send_sge = resp.max_send_sge; + init_attr->cap.max_recv_sge = resp.max_recv_sge; + init_attr->cap.max_inline_data = resp.max_inline_data; + init_attr->sq_sig_all = resp.sq_sig_all; + + return 0; +} + static int ibv_cmd_destroy_qp_v1(struct ibv_qp *qp) { struct ibv_destroy_qp_v1 cmd; @@ -1192,3 +1374,36 @@ int ibv_cmd_close_xrc_domain(struct ibv_xrc_domain *d) return 0; } +int ibv_cmd_reg_xrc_rcv_qp(struct ibv_xrc_domain *d, uint32_t xrc_qp_num) +{ + struct ibv_reg_xrc_rcv_qp cmd; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD(&cmd, sizeof cmd, REG_XRC_RCV_QP); + cmd.xrc_domain_handle = d->handle; + cmd.qp_num = xrc_qp_num; + + if (write(d->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) + return errno; + return 0; +} + +int ibv_cmd_unreg_xrc_rcv_qp(struct ibv_xrc_domain *d, uint32_t xrc_qp_num) +{ + struct ibv_unreg_xrc_rcv_qp cmd; + + if (abi_ver < 6) + return ENOSYS; + + IBV_INIT_CMD(&cmd, sizeof cmd, UNREG_XRC_RCV_QP); + cmd.xrc_domain_handle = d->handle; + cmd.qp_num = xrc_qp_num; + + if (write(d->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) + return errno; + return 0; +} + + diff --git a/src/device.c b/src/device.c index 3abc1eb..55ed3d7 100644 --- a/src/device.c +++ b/src/device.c @@ -182,31 +182,33 @@ int __ibv_get_async_event(struct ibv_context *context, event->event_type = ev.event_type; - switch (event->event_type) { - case IBV_EVENT_CQ_ERR: - event->element.cq = (void *) (uintptr_t) ev.element; - break; - - case IBV_EVENT_QP_FATAL: - case IBV_EVENT_QP_REQ_ERR: - case IBV_EVENT_QP_ACCESS_ERR: - case IBV_EVENT_COMM_EST: - case IBV_EVENT_SQ_DRAINED: - case IBV_EVENT_PATH_MIG: - case IBV_EVENT_PATH_MIG_ERR: - case IBV_EVENT_QP_LAST_WQE_REACHED: - event->element.qp = (void *) (uintptr_t) ev.element; - break; - - case IBV_EVENT_SRQ_ERR: - case IBV_EVENT_SRQ_LIMIT_REACHED: - event->element.srq = (void *) (uintptr_t) ev.element; - break; - - default: - event->element.port_num = ev.element; - break; - } + if (event->event_type & IBV_XRC_QP_EVENT_FLAG) { + event->element.xrc_qp_num = ev.element; + } else + switch (event->event_type) { + case IBV_EVENT_CQ_ERR: + event->element.cq = (void *) (uintptr_t) ev.element; + break; + + case IBV_EVENT_QP_FATAL: + case IBV_EVENT_QP_REQ_ERR: + case IBV_EVENT_QP_ACCESS_ERR: + case IBV_EVENT_COMM_EST: + case IBV_EVENT_SQ_DRAINED: + case IBV_EVENT_PATH_MIG: + case IBV_EVENT_PATH_MIG_ERR: + case IBV_EVENT_QP_LAST_WQE_REACHED: + event->element.qp = (void *) (uintptr_t) ev.element; + break; + + case IBV_EVENT_SRQ_ERR: + case IBV_EVENT_SRQ_LIMIT_REACHED: + event->element.srq = (void *) (uintptr_t) ev.element; + break; + default: + event->element.port_num = ev.element; + break; + } if (context->ops.async_event) context->ops.async_event(event); diff --git a/src/libibverbs.map b/src/libibverbs.map index d1a6a47..fce6965 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -97,6 +97,16 @@ IBVERBS_1.1 { ibv_cmd_open_xrc_domain; ibv_close_xrc_domain; ibv_cmd_close_xrc_domain; + ibv_create_xrc_rcv_qp; + ibv_cmd_create_xrc_rcv_qp; + ibv_modify_xrc_rcv_qp; + ibv_cmd_modify_xrc_rcv_qp; + ibv_query_xrc_rcv_qp; + ibv_cmd_query_xrc_rcv_qp; + ibv_reg_xrc_rcv_qp; + ibv_cmd_reg_xrc_rcv_qp; + ibv_unreg_xrc_rcv_qp; + ibv_cmd_unreg_xrc_rcv_qp; ibv_node_type_str; ibv_port_state_str; diff --git a/src/verbs.c b/src/verbs.c index f09542f..bfc6805 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -597,3 +597,62 @@ int ibv_close_xrc_domain(struct ibv_xrc_domain *d) return d->context->more_ops->close_xrc_domain(d); } + +int ibv_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, + uint32_t *xrc_rcv_qpn) +{ + struct ibv_context *c; + if (!init_attr || !(init_attr->xrc_domain)) + return EINVAL; + + c = init_attr->xrc_domain->context; + if (!c->more_ops) + return ENOSYS; + + return c->more_ops->create_xrc_rcv_qp(init_attr, + xrc_rcv_qpn); +} + +int ibv_modify_xrc_rcv_qp(struct ibv_xrc_domain *d, + uint32_t xrc_rcv_qpn, + struct ibv_qp_attr *attr, + int attr_mask) +{ + if (!d || !attr) + return EINVAL; + + if (!d->context->more_ops) + return ENOSYS; + + return d->context->more_ops->modify_xrc_rcv_qp(d, xrc_rcv_qpn, attr, + attr_mask); +} + +int ibv_query_xrc_rcv_qp(struct ibv_xrc_domain *d, + uint32_t xrc_rcv_qpn, + struct ibv_qp_attr *attr, + int attr_mask, + struct ibv_qp_init_attr *init_attr) +{ + if (!d) + return EINVAL; + + if (!d->context->more_ops) + return ENOSYS; + + return d->context->more_ops->query_xrc_rcv_qp(d, xrc_rcv_qpn, attr, + attr_mask, init_attr); +} + +int ibv_reg_xrc_rcv_qp(struct ibv_xrc_domain *d, + uint32_t xrc_rcv_qpn) +{ + return d->context->more_ops->reg_xrc_rcv_qp(d, xrc_rcv_qpn); +} + +int ibv_unreg_xrc_rcv_qp(struct ibv_xrc_domain *d, + uint32_t xrc_rcv_qpn) +{ + return d->context->more_ops->unreg_xrc_rcv_qp(d, xrc_rcv_qpn); +} + -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:29 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:29 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 3/12 V2] libmlx4: XRC base implementation Message-ID: <200807101851.30229.jackm@dev.mellanox.co.il> From 33c27150ed20ff34622a45abd5ff27727f4c34aa Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Wed, 9 Jul 2008 13:55:24 +0300 Subject: [PATCH] XRC implementation, consolidated: xrc ops were moved to their own structure at the end of struct ibv_context (to preserve binary compatibility). Check for ibv_context.xrc_ops member via AC_CHECK_MEMBER XRC QPs have MSB set in qp number, for identification in completion handling. jackm at mtldesk006:/local/roland/libmlx4> head -n 20 patches/xrc_consolidated.patch XRC implementation, consolidated: xrc ops were moved to their own structure at the end of struct ibv_context (to preserve binary compatibility). Check for ibv_context.xrc_ops member via AC_CHECK_MEMBER XRC QPs have MSB set in qp number, for identification in completion handling. Various bug fixes. (OFED 1.3 commit 39fe7f47e8fc07f356098df048d00740ba585fc5) V2: 1. checkpatch.pl cleanup 2. Changed xrc_ops to more ops 3. Check for xrc verbs in ibv_more_ops via AC_CHECK_MEMBER Signed-off-by: Jack Morgenstein --- configure.in | 3 + src/cq.c | 35 ++++++++++++-- src/mlx4-abi.h | 16 ++++++ src/mlx4.c | 20 ++++++++ src/mlx4.h | 46 ++++++++++++++++++ src/qp.c | 7 ++- src/srq.c | 50 +++++++++++++++++++ src/verbs.c | 144 +++++++++++++++++++++++++++++++++++++++++++++++++++----- src/wqe.h | 2 +- 9 files changed, 303 insertions(+), 20 deletions(-) diff --git a/configure.in b/configure.in index 25f27f7..9304539 100644 --- a/configure.in +++ b/configure.in @@ -42,6 +42,12 @@ AC_CHECK_HEADER(valgrind/memcheck.h, dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_CHECK_SIZEOF(long) +AC_CHECK_MEMBER(struct ibv_context.more_ops, + [AC_DEFINE([HAVE_IBV_MORE_OPS], 1, [Define to 1 if more_ops is a member of ibv_context])],, + [#include ]) +AC_CHECK_MEMBER(struct ibv_more_ops.create_xrc_srq, + [AC_DEFINE([HAVE_IBV_XRC_OPS], 1, [Define to 1 if have xrc ops])],, + [#include ]) dnl Checks for library functions AC_CHECK_FUNC(ibv_read_sysfs_file, [], diff --git a/src/cq.c b/src/cq.c index 68e16e9..c598b87 100644 --- a/src/cq.c +++ b/src/cq.c @@ -196,10 +196,12 @@ static int mlx4_poll_one(struct mlx4_cq *cq, struct mlx4_cqe *cqe; struct mlx4_srq *srq; uint32_t qpn; + uint32_t srqn; uint32_t g_mlpath_rqpn; uint16_t wqe_index; int is_error; int is_send; + int is_src_recv = 0; cqe = next_cqe_sw(cq); if (!cqe) @@ -221,20 +223,30 @@ static int mlx4_poll_one(struct mlx4_cq *cq, is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_ERROR; - if (!*cur_qp || - (ntohl(cqe->my_qpn) & 0xffffff) != (*cur_qp)->ibv_qp.qp_num) { + if (qpn & MLX4_XRC_QPN_BIT && !is_send) { + srqn = ntohl(cqe->g_mlpath_rqpn) & 0xffffff; + /* + * We do not have to take the XRC SRQ table lock here, + * because CQs will be locked while XRC SRQs are removed + * from the table. + */ + srq = mlx4_find_xrc_srq(to_mctx(cq->ibv_cq.context), srqn); + if (!srq) + return CQ_POLL_ERR; + is_src_recv = 1; + } else if (!*cur_qp || (qpn & 0xffffff) != (*cur_qp)->ibv_qp.qp_num) { /* * We do not have to take the QP table lock here, * because CQs will be locked while QPs are removed * from the table. */ *cur_qp = mlx4_find_qp(to_mctx(cq->ibv_cq.context), - ntohl(cqe->my_qpn) & 0xffffff); + qpn & 0xffffff); if (!*cur_qp) return CQ_POLL_ERR; } - wc->qp_num = (*cur_qp)->ibv_qp.qp_num; + wc->qp_num = qpn & 0xffffff; if (is_send) { wq = &(*cur_qp)->sq; @@ -242,6 +254,10 @@ static int mlx4_poll_one(struct mlx4_cq *cq, wq->tail += (uint16_t) (wqe_index - (uint16_t) wq->tail); wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)]; ++wq->tail; + } else if (is_src_recv) { + wqe_index = htons(cqe->wqe_index); + wc->wr_id = srq->wrid[wqe_index]; + mlx4_free_srq_wqe(srq, wqe_index); } else if ((*cur_qp)->ibv_qp.srq) { srq = to_msrq((*cur_qp)->ibv_qp.srq); wqe_index = htons(cqe->wqe_index); @@ -387,6 +403,10 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) uint32_t prod_index; uint8_t owner_bit; int nfreed = 0; + int is_xrc_srq = 0; + + if (srq && srq->ibv_srq.xrc_cq) + is_xrc_srq = 1; /* * First we need to find the current producer index, so we @@ -405,7 +425,12 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) */ while ((int) --prod_index - (int) cq->cons_index >= 0) { cqe = get_cqe(cq, prod_index & cq->ibv_cq.cqe); - if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { + if (is_xrc_srq && + (ntohl(cqe->g_mlpath_rqpn & 0xffffff) == srq->srqn) && + !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) { + mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); + ++nfreed; + } else if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { if (srq && !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); ++nfreed; diff --git a/src/mlx4-abi.h b/src/mlx4-abi.h index 20a40c9..1b1253c 100644 --- a/src/mlx4-abi.h +++ b/src/mlx4-abi.h @@ -68,6 +68,14 @@ struct mlx4_resize_cq { __u64 buf_addr; }; +#ifdef HAVE_IBV_XRC_OPS +struct mlx4_create_xrc_srq { + struct ibv_create_xrc_srq ibv_cmd; + __u64 buf_addr; + __u64 db_addr; +}; +#endif + struct mlx4_create_srq { struct ibv_create_srq ibv_cmd; __u64 buf_addr; @@ -90,4 +98,12 @@ struct mlx4_create_qp { __u8 reserved[5]; }; +#ifdef HAVE_IBV_XRC_OPS +struct mlx4_open_xrc_domain_resp { + struct ibv_open_xrc_domain_resp ibv_resp; + __u32 xrcdn; + __u32 reserved; +}; +#endif + #endif /* MLX4_ABI_H */ diff --git a/src/mlx4.c b/src/mlx4.c index 34ece39..8a46543 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -68,6 +68,16 @@ struct { HCA(MELLANOX, 0x673c), /* MT25408 "Hermon" QDR PCIe gen2 */ }; +#ifdef HAVE_IBV_MORE_OPS +static struct ibv_more_ops mlx4_more_ops = { +#ifdef HAVE_IBV_XRC_OPS + .create_xrc_srq = mlx4_create_xrc_srq, + .open_xrc_domain = mlx4_open_xrc_domain, + .close_xrc_domain = mlx4_close_xrc_domain, +#endif +}; +#endif + static struct ibv_context_ops mlx4_ctx_ops = { .query_device = mlx4_query_device, .query_port = mlx4_query_port, @@ -123,6 +131,15 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ for (i = 0; i < MLX4_QP_TABLE_SIZE; ++i) context->qp_table[i].refcnt = 0; + context->num_xrc_srqs = resp.qp_tab_size; + context->xrc_srq_table_shift = ffs(context->num_xrc_srqs) - 1 + - MLX4_XRC_SRQ_TABLE_BITS; + context->xrc_srq_table_mask = (1 << context->xrc_srq_table_shift) - 1; + + pthread_mutex_init(&context->xrc_srq_table_mutex, NULL); + for (i = 0; i < MLX4_XRC_SRQ_TABLE_SIZE; ++i) + context->xrc_srq_table[i].refcnt = 0; + for (i = 0; i < MLX4_NUM_DB_TYPE; ++i) context->db_list[i] = NULL; @@ -155,6 +172,9 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ pthread_spin_init(&context->uar_lock, PTHREAD_PROCESS_PRIVATE); context->ibv_ctx.ops = mlx4_ctx_ops; +#ifdef HAVE_IBV_XRC_OPS + context->ibv_ctx.more_ops = &mlx4_more_ops; +#endif return &context->ibv_ctx; diff --git a/src/mlx4.h b/src/mlx4.h index 827a201..28a8f9c 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -79,6 +79,11 @@ #endif +#ifndef HAVE_IBV_MORE_OPS +#undef HAVE_IBV_XRC_OPS +#undef HAVE_IBV_CREATE_QP_EXP +#endif + #define HIDDEN __attribute__((visibility ("hidden"))) #define PFX "mlx4: " @@ -97,6 +97,16 @@ enum { MLX4_QP_TABLE_MASK = MLX4_QP_TABLE_SIZE - 1 }; +enum { + MLX4_XRC_SRQ_TABLE_BITS = 8, + MLX4_XRC_SRQ_TABLE_SIZE = 1 << MLX4_XRC_SRQ_TABLE_BITS, + MLX4_XRC_SRQ_TABLE_MASK = MLX4_XRC_SRQ_TABLE_SIZE - 1 +}; + +enum { + MLX4_XRC_QPN_BIT = (1 << 23) +}; + enum mlx4_db_type { MLX4_DB_TYPE_CQ, MLX4_DB_TYPE_RQ, @@ -157,6 +167,15 @@ struct mlx4_context { int qp_table_shift; int qp_table_mask; + struct { + struct mlx4_srq **table; + int refcnt; + } xrc_srq_table[MLX4_XRC_SRQ_TABLE_SIZE]; + pthread_mutex_t xrc_srq_table_mutex; + int num_xrc_srqs; + int xrc_srq_table_shift; + int xrc_srq_table_mask; + struct mlx4_db_page *db_list[MLX4_NUM_DB_TYPE]; pthread_mutex_t db_list_mutex; }; @@ -243,6 +262,11 @@ struct mlx4_ah { struct mlx4_av av; }; +struct mlx4_xrc_domain { + struct ibv_xrc_domain ibv_xrcd; + uint32_t xrcdn; +}; + static inline unsigned long align(unsigned long val, unsigned long align) { return (val + align - 1) & ~(align - 1); @@ -287,6 +311,13 @@ static inline struct mlx4_ah *to_mah(struct ibv_ah *ibah) return to_mxxx(ah, ah); } +#ifdef HAVE_IBV_XRC_OPS +static inline struct mlx4_xrc_domain *to_mxrcd(struct ibv_xrc_domain *ibxrcd) +{ + return to_mxxx(xrcd, xrc_domain); +} +#endif + int mlx4_alloc_buf(struct mlx4_buf *buf, size_t size, int page_size); void mlx4_free_buf(struct mlx4_buf *buf); @@ -333,6 +364,10 @@ void mlx4_free_srq_wqe(struct mlx4_srq *srq, int ind); int mlx4_post_srq_recv(struct ibv_srq *ibsrq, struct ibv_recv_wr *wr, struct ibv_recv_wr **bad_wr); +struct mlx4_srq *mlx4_find_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn); +int mlx4_store_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn, + struct mlx4_srq *srq); +void mlx4_clear_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn); struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr); int mlx4_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, @@ -363,5 +398,16 @@ int mlx4_alloc_av(struct mlx4_pd *pd, struct ibv_ah_attr *attr, void mlx4_free_av(struct mlx4_ah *ah); int mlx4_attach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); int mlx4_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); +#ifdef HAVE_IBV_XRC_OPS +struct ibv_srq *mlx4_create_xrc_srq(struct ibv_pd *pd, + struct ibv_xrc_domain *xrc_domain, + struct ibv_cq *xrc_cq, + struct ibv_srq_init_attr *attr); +struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, + int fd, int oflag); + +int mlx4_close_xrc_domain(struct ibv_xrc_domain *d); +#endif + #endif /* MLX4_H */ diff --git a/src/qp.c b/src/qp.c index bb98c09..02c5d34 100644 --- a/src/qp.c +++ b/src/qp.c @@ -226,7 +226,7 @@ int mlx4_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, ctrl = wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1)); qp->sq.wrid[ind & (qp->sq.wqe_cnt - 1)] = wr->wr_id; - ctrl->srcrb_flags = + ctrl->xrcrb_flags = (wr->send_flags & IBV_SEND_SIGNALED ? htonl(MLX4_WQE_CTRL_CQ_UPDATE) : 0) | (wr->send_flags & IBV_SEND_SOLICITED ? @@ -243,6 +243,9 @@ int mlx4_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, size = sizeof *ctrl / 16; switch (ibqp->qp_type) { + case IBV_QPT_XRC: + ctrl->xrcrb_flags |= htonl(wr->xrc_remote_srq_num << 8); + /* fall thru */ case IBV_QPT_RC: case IBV_QPT_UC: switch (wr->opcode) { @@ -543,6 +546,7 @@ void mlx4_calc_sq_wqe_size(struct ibv_qp_cap *cap, enum ibv_qp_type type, size += sizeof (struct mlx4_wqe_raddr_seg); break; + case IBV_QPT_XRC: case IBV_QPT_RC: size += sizeof (struct mlx4_wqe_raddr_seg); /* @@ -630,6 +634,7 @@ void mlx4_set_sq_sizes(struct mlx4_qp *qp, struct ibv_qp_cap *cap, case IBV_QPT_UC: case IBV_QPT_RC: + case IBV_QPT_XRC: wqe_size -= sizeof (struct mlx4_wqe_raddr_seg); break; diff --git a/src/srq.c b/src/srq.c index ba2ceb9..1350792 100644 --- a/src/srq.c +++ b/src/srq.c @@ -167,3 +167,53 @@ int mlx4_alloc_srq_buf(struct ibv_pd *pd, struct ibv_srq_attr *attr, return 0; } + +struct mlx4_srq *mlx4_find_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn) +{ + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; + + if (ctx->xrc_srq_table[tind].refcnt) + return ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask]; + else + return NULL; +} + +int mlx4_store_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn, + struct mlx4_srq *srq) +{ + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; + int ret = 0; + + pthread_mutex_lock(&ctx->xrc_srq_table_mutex); + + if (!ctx->xrc_srq_table[tind].refcnt) { + ctx->xrc_srq_table[tind].table = calloc(ctx->xrc_srq_table_mask + 1, + sizeof(struct mlx4_srq *)); + if (!ctx->xrc_srq_table[tind].table) { + ret = -1; + goto out; + } + } + + ++ctx->xrc_srq_table[tind].refcnt; + ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask] = srq; + +out: + pthread_mutex_unlock(&ctx->xrc_srq_table_mutex); + return ret; +} + +void mlx4_clear_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn) +{ + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; + + pthread_mutex_lock(&ctx->xrc_srq_table_mutex); + + if (!--ctx->xrc_srq_table[tind].refcnt) + free(ctx->xrc_srq_table[tind].table); + else + ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask] = NULL; + + pthread_mutex_unlock(&ctx->xrc_srq_table_mutex); +} + diff --git a/src/verbs.c b/src/verbs.c index 3935ed5..b883992 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -368,18 +368,36 @@ int mlx4_query_srq(struct ibv_srq *srq, return ibv_cmd_query_srq(srq, attr, &cmd, sizeof cmd); } -int mlx4_destroy_srq(struct ibv_srq *srq) +int mlx4_destroy_srq(struct ibv_srq *ibsrq) { + struct mlx4_srq *srq = to_msrq(ibsrq); + struct mlx4_cq *mcq = NULL; int ret; - ret = ibv_cmd_destroy_srq(srq); - if (ret) + if (ibsrq->xrc_cq) { + /* is an xrc_srq */ + mcq = to_mcq(ibsrq->xrc_cq); + mlx4_cq_clean(mcq, 0, srq); + pthread_spin_lock(&mcq->lock); + mlx4_clear_xrc_srq(to_mctx(ibsrq->context), srq->srqn); + pthread_spin_unlock(&mcq->lock); + } + + ret = ibv_cmd_destroy_srq(ibsrq); + if (ret) { + if (ibsrq->xrc_cq) { + pthread_spin_lock(&mcq->lock); + mlx4_store_xrc_srq(to_mctx(ibsrq->context), + srq->srqn, srq); + pthread_spin_unlock(&mcq->lock); + } return ret; + } - mlx4_free_db(to_mctx(srq->context), MLX4_DB_TYPE_RQ, to_msrq(srq)->db); - mlx4_free_buf(&to_msrq(srq)->buf); - free(to_msrq(srq)->wrid); - free(to_msrq(srq)); + mlx4_free_db(to_mctx(ibsrq->context), MLX4_DB_TYPE_RQ, srq->db); + mlx4_free_buf(&srq->buf); + free(srq->wrid); + free(srq); return 0; } @@ -413,7 +431,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); - if (attr->srq) + if (attr->srq || attr->qp_type == IBV_QPT_XRC) attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; else { if (attr->cap.max_recv_sge < 1) @@ -431,7 +449,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) goto err_free; - if (!attr->srq) { + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); if (!qp->db) goto err_free; @@ -440,7 +458,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) } cmd.buf_addr = (uintptr_t) qp->buf.buf; - if (attr->srq) + if (attr->srq || attr->qp_type == IBV_QPT_XRC) cmd.db_addr = 0; else cmd.db_addr = (uintptr_t) qp->db; @@ -477,7 +495,7 @@ err_destroy: ibv_cmd_destroy_qp(&qp->ibv_qp); err_rq_db: - if (!attr->srq) + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); err_free: @@ -536,7 +554,7 @@ int mlx4_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, mlx4_cq_clean(to_mcq(qp->send_cq), qp->qp_num, NULL); mlx4_init_qp_indices(to_mqp(qp)); - if (!qp->srq) + if (!qp->srq && qp->qp_type != IBV_QPT_XRC) *to_mqp(qp)->db = 0; } @@ -595,7 +613,7 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) mlx4_unlock_cqs(ibqp); - if (!ibqp->srq) + if (!ibqp->srq && ibqp->qp_type != IBV_QPT_XRC) mlx4_free_db(to_mctx(ibqp->context), MLX4_DB_TYPE_RQ, qp->db); free(qp->sq.wrid); if (qp->rq.wqe_cnt) @@ -653,3 +671,103 @@ int mlx4_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) { return ibv_cmd_detach_mcast(qp, gid, lid); } + +#ifdef HAVE_IBV_XRC_OPS +struct ibv_srq *mlx4_create_xrc_srq(struct ibv_pd *pd, + struct ibv_xrc_domain *xrc_domain, + struct ibv_cq *xrc_cq, + struct ibv_srq_init_attr *attr) +{ + struct mlx4_create_xrc_srq cmd; + struct mlx4_create_srq_resp resp; + struct mlx4_srq *srq; + int ret; + + /* Sanity check SRQ size before proceeding */ + if (attr->attr.max_wr > 1 << 16 || attr->attr.max_sge > 64) + return NULL; + + srq = malloc(sizeof *srq); + if (!srq) + return NULL; + + if (pthread_spin_init(&srq->lock, PTHREAD_PROCESS_PRIVATE)) + goto err; + + srq->max = align_queue_size(attr->attr.max_wr + 1); + srq->max_gs = attr->attr.max_sge; + srq->counter = 0; + + if (mlx4_alloc_srq_buf(pd, &attr->attr, srq)) + goto err; + + srq->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); + if (!srq->db) + goto err_free; + + *srq->db = 0; + + cmd.buf_addr = (uintptr_t) srq->buf.buf; + cmd.db_addr = (uintptr_t) srq->db; + + ret = ibv_cmd_create_xrc_srq(pd, &srq->ibv_srq, attr, + xrc_domain->handle, + xrc_cq->handle, + &cmd.ibv_cmd, sizeof cmd, + &resp.ibv_resp, sizeof resp); + if (ret) + goto err_db; + + srq->ibv_srq.xrc_srq_num = srq->srqn = resp.srqn; + + ret = mlx4_store_xrc_srq(to_mctx(pd->context), srq->ibv_srq.xrc_srq_num, srq); + if (ret) + goto err_destroy; + + return &srq->ibv_srq; + +err_destroy: + ibv_cmd_destroy_srq(&srq->ibv_srq); + +err_db: + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, srq->db); + +err_free: + free(srq->wrid); + mlx4_free_buf(&srq->buf); + +err: + free(srq); + + return NULL; +} + +struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, + int fd, int oflag) +{ + int ret; + struct mlx4_open_xrc_domain_resp resp; + struct mlx4_xrc_domain *xrcd; + + xrcd = malloc(sizeof *xrcd); + if (!xrcd) + return NULL; + + ret = ibv_cmd_open_xrc_domain(context, fd, oflag, &xrcd->ibv_xrcd, + &resp.ibv_resp, sizeof resp); + if (ret) { + free(xrcd); + return NULL; + } + + xrcd->xrcdn = resp.xrcdn; + return &xrcd->ibv_xrcd; +} + +int mlx4_close_xrc_domain(struct ibv_xrc_domain *d) +{ + ibv_cmd_close_xrc_domain(d); + free(d); + return 0; +} +#endif diff --git a/src/wqe.h b/src/wqe.h index 6f7f309..fa2f8ac 100644 --- a/src/wqe.h +++ b/src/wqe.h @@ -65,7 +65,7 @@ struct mlx4_wqe_ctrl_seg { * [1] SE (solicited event) * [0] FL (force loopback) */ - uint32_t srcrb_flags; + uint32_t xrcrb_flags; /* * imm is immediate data for send/RDMA write w/ immediate; * also invalidation key for send with invalidate; input -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:49 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:49 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 8/12 V2] core: XRC receive-only QPs Message-ID: <200807101851.49575.jackm@dev.mellanox.co.il> From f4059b73d35e3836d7977f1f6135bed5f9123123 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Tue, 8 Jul 2008 10:11:50 +0300 Subject: [PATCH] IB/core: Implement XRC receive-only QPs for userspace apps. Added creation of XRC receive-only QPs for userspace, which reside in kernel space (user cannot post-to or poll these QPs). Motivation: MPI community requires XRC receive QPs which will not be destroyed when the creating process terminates. Solution: Userspace requests that a QP be created in kernel space. Each userspace process using that QP (i.e. receiving packets on an XRC SRQ via the qp), registers with that QP (-- the creator is also registered, whether or not it is a user of the QP). When the last userspace user unregisters with the QP, it is destroyed. Unregistration is also part of userspace process cleanup, so there is no leakage. This patch implements the kernel procedures to implement the following (new) libibverbs API: ibv_create_xrc_rcv_qp ibv_modify_xrc_rcv_qp ibv_query_xrc_rcv_qp ibv_reg_xrc_rcv_qp ibv_unreg_xrc_rcv_qp In addition, the patch implements the foundation for distributing XRC-receive-only QP events to userspace processes registered with that QP. Finally, the patch modifies ib_uverbs_close_xrc_domain() to return BUSY if any resources are still in use by the process, so that the XRC rcv-only QP cleanup can operate properly. V2: 1. checkpatch.pl cleanups 2. eliminated XRC_RCV create-qp flag 3. Moved list of registered QPs from general context to individual XRC domains. (ib_uxrcd_object structure) 4. Added locking to ib_uverbs_xrc_rcv_modify_qp() and ib_uverbs_xrc_rcv_query_qp() 5. Fixed alignment problems in xrc_rcv ABI Signed-off-by: Jack Morgenstein --- drivers/infiniband/core/uverbs.h | 14 ++ drivers/infiniband/core/uverbs_cmd.c | 372 ++++++++++++++++++++++++++++++++- drivers/infiniband/core/uverbs_main.c | 23 ++ include/rdma/ib_user_verbs.h | 78 +++++++- include/rdma/ib_verbs.h | 28 +++ 5 files changed, 511 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index df9b6d2..b55f0d7 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -133,6 +133,11 @@ struct ib_ucq_object { u32 async_events_reported; }; +struct ib_uxrcd_object { + struct ib_uobject uobject; + struct list_head xrc_reg_qp_list; +}; + extern spinlock_t ib_uverbs_idr_lock; extern struct idr ib_uverbs_pd_idr; extern struct idr ib_uverbs_mr_idr; @@ -161,8 +166,12 @@ void ib_uverbs_qp_event_handler(struct ib_event *event, void *context_ptr); void ib_uverbs_srq_event_handler(struct ib_event *event, void *context_ptr); void ib_uverbs_event_handler(struct ib_event_handler *handler, struct ib_event *event); +void ib_uverbs_xrc_rcv_qp_event_handler(struct ib_event *event, + void *context_ptr); void ib_uverbs_dealloc_xrcd(struct ib_device *ib_dev, struct ib_xrcd *xrcd); +int ib_uverbs_cleanup_xrc_rcv_qp(struct ib_uverbs_file *file, + struct ib_xrcd *xrcd, u32 qp_num); #define IB_UVERBS_DECLARE_CMD(name) \ ssize_t ib_uverbs_##name(struct ib_uverbs_file *file, \ @@ -200,6 +209,11 @@ IB_UVERBS_DECLARE_CMD(destroy_srq); IB_UVERBS_DECLARE_CMD(create_xrc_srq); IB_UVERBS_DECLARE_CMD(open_xrc_domain); IB_UVERBS_DECLARE_CMD(close_xrc_domain); +IB_UVERBS_DECLARE_CMD(create_xrc_rcv_qp); +IB_UVERBS_DECLARE_CMD(modify_xrc_rcv_qp); +IB_UVERBS_DECLARE_CMD(query_xrc_rcv_qp); +IB_UVERBS_DECLARE_CMD(reg_xrc_rcv_qp); +IB_UVERBS_DECLARE_CMD(unreg_xrc_rcv_qp); #endif /* UVERBS_H */ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 6b333ca..4402a07 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1078,6 +1078,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, goto err_put; } + attr.create_flags = 0; attr.event_handler = ib_uverbs_qp_event_handler; attr.qp_context = file; attr.send_cq = scq; @@ -2428,6 +2429,7 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, struct ib_uverbs_open_xrc_domain_resp resp; struct ib_udata udata; struct ib_uobject *uobj; + struct ib_uxrcd_object *xrcd_uobj; struct ib_xrcd *xrcd = NULL; struct file *f = NULL; struct inode *inode = NULL; @@ -2472,12 +2474,13 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, } } - uobj = kmalloc(sizeof *uobj, GFP_KERNEL); - if (!uobj) { + xrcd_uobj = kmalloc(sizeof *xrcd_uobj, GFP_KERNEL); + if (!xrcd_uobj) { ret = -ENOMEM; goto err_table_mutex_unlock; } + uobj = &xrcd_uobj->uobject; init_uobj(uobj, 0, file->ucontext, &pd_lock_key); down_write(&uobj->mutex); @@ -2521,6 +2524,8 @@ ssize_t ib_uverbs_open_xrc_domain(struct ib_uverbs_file *file, goto err_copy; } + INIT_LIST_HEAD(&xrcd_uobj->xrc_reg_qp_list); + mutex_lock(&file->mutex); list_add_tail(&uobj->list, &file->ucontext->xrc_domain_list); mutex_unlock(&file->mutex); @@ -2563,6 +2568,7 @@ ssize_t ib_uverbs_close_xrc_domain(struct ib_uverbs_file *file, { struct ib_uverbs_close_xrc_domain cmd; struct ib_uobject *uobj; + struct ib_uxrcd_object *xrcd_uobj; struct ib_xrcd *xrcd = NULL; struct inode *inode = NULL; int ret = 0; @@ -2578,6 +2584,13 @@ ssize_t ib_uverbs_close_xrc_domain(struct ib_uverbs_file *file, goto err_unlock_mutex; } + xrcd_uobj = container_of(uobj, struct ib_uxrcd_object, uobject); + if (!list_empty(&xrcd_uobj->xrc_reg_qp_list)) { + ret = -EBUSY; + put_uobj_write(uobj); + goto err_unlock_mutex; + } + xrcd = (struct ib_xrcd *) (uobj->object); inode = xrcd->inode; @@ -2613,7 +2626,7 @@ err_unlock_mutex: } void ib_uverbs_dealloc_xrcd(struct ib_device *ib_dev, - struct ib_xrcd *xrcd) + struct ib_xrcd *xrcd) { struct inode *inode = NULL; int ret = 0; @@ -2627,4 +2640,357 @@ void ib_uverbs_dealloc_xrcd(struct ib_device *ib_dev, xrcd_table_delete(ib_dev, inode); } +ssize_t ib_uverbs_create_xrc_rcv_qp(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_create_xrc_rcv_qp cmd; + struct ib_uverbs_create_xrc_rcv_qp_resp resp; + struct ib_uxrc_rcv_object *obj; + struct ib_qp_init_attr init_attr; + struct ib_xrcd *xrcd; + struct ib_uobject *uobj; + struct ib_uxrcd_object *xrcd_uobj; + u32 qp_num; + int err; + + if (out_len < sizeof resp) + return -ENOSPC; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + obj = kzalloc(sizeof *obj, GFP_KERNEL); + if (!obj) + return -ENOMEM; + + xrcd = idr_read_xrcd(cmd.xrc_domain_handle, file->ucontext, &uobj); + if (!xrcd) { + err = -EINVAL; + goto err_out; + } + init_attr.event_handler = ib_uverbs_xrc_rcv_qp_event_handler; + init_attr.qp_context = file; + init_attr.srq = NULL; + init_attr.sq_sig_type = + cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; + init_attr.qp_type = IB_QPT_XRC; + init_attr.xrc_domain = xrcd; + + init_attr.cap.max_send_wr = 1; + init_attr.cap.max_recv_wr = 0; + init_attr.cap.max_send_sge = 1; + init_attr.cap.max_recv_sge = 0; + init_attr.cap.max_inline_data = 0; + + err = xrcd->device->create_xrc_rcv_qp(&init_attr, &qp_num); + if (err) + goto err_put; + + memset(&resp, 0, sizeof resp); + resp.qpn = qp_num; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + err = -EFAULT; + goto err_destroy; + } + + atomic_inc(&xrcd->usecnt); + put_xrcd_read(uobj); + obj->qp_num = qp_num; + obj->domain_handle = cmd.xrc_domain_handle; + xrcd_uobj = container_of(uobj, struct ib_uxrcd_object, uobject); + mutex_lock(&file->device->ib_dev->xrcd_table_mutex); + list_add_tail(&obj->list, &xrcd_uobj->xrc_reg_qp_list); + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); + + return in_len; + +err_destroy: + xrcd->device->unreg_xrc_rcv_qp(xrcd, file, qp_num); +err_put: + put_xrcd_read(uobj); +err_out: + kfree(obj); + return err; +} + +ssize_t ib_uverbs_modify_xrc_rcv_qp(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_modify_xrc_rcv_qp cmd; + struct ib_qp_attr *attr; + struct ib_xrcd *xrcd; + struct ib_uobject *uobj; + int err; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + attr = kzalloc(sizeof *attr, GFP_KERNEL); + if (!attr) + return -ENOMEM; + + xrcd = idr_read_xrcd(cmd.xrc_domain_handle, file->ucontext, &uobj); + if (!xrcd) { + kfree(attr); + return -EINVAL; + } + + attr->qp_state = cmd.qp_state; + attr->cur_qp_state = cmd.cur_qp_state; + attr->qp_access_flags = cmd.qp_access_flags; + attr->pkey_index = cmd.pkey_index; + attr->port_num = cmd.port_num; + attr->path_mtu = cmd.path_mtu; + attr->path_mig_state = cmd.path_mig_state; + attr->qkey = cmd.qkey; + attr->rq_psn = cmd.rq_psn; + attr->sq_psn = cmd.sq_psn; + attr->dest_qp_num = cmd.dest_qp_num; + attr->alt_pkey_index = cmd.alt_pkey_index; + attr->en_sqd_async_notify = cmd.en_sqd_async_notify; + attr->max_rd_atomic = cmd.max_rd_atomic; + attr->max_dest_rd_atomic = cmd.max_dest_rd_atomic; + attr->min_rnr_timer = cmd.min_rnr_timer; + attr->port_num = cmd.port_num; + attr->timeout = cmd.timeout; + attr->retry_cnt = cmd.retry_cnt; + attr->rnr_retry = cmd.rnr_retry; + attr->alt_port_num = cmd.alt_port_num; + attr->alt_timeout = cmd.alt_timeout; + + memcpy(attr->ah_attr.grh.dgid.raw, cmd.dest.dgid, 16); + attr->ah_attr.grh.flow_label = cmd.dest.flow_label; + attr->ah_attr.grh.sgid_index = cmd.dest.sgid_index; + attr->ah_attr.grh.hop_limit = cmd.dest.hop_limit; + attr->ah_attr.grh.traffic_class = cmd.dest.traffic_class; + attr->ah_attr.dlid = cmd.dest.dlid; + attr->ah_attr.sl = cmd.dest.sl; + attr->ah_attr.src_path_bits = cmd.dest.src_path_bits; + attr->ah_attr.static_rate = cmd.dest.static_rate; + attr->ah_attr.ah_flags = cmd.dest.is_global ? IB_AH_GRH : 0; + attr->ah_attr.port_num = cmd.dest.port_num; + + memcpy(attr->alt_ah_attr.grh.dgid.raw, cmd.alt_dest.dgid, 16); + attr->alt_ah_attr.grh.flow_label = cmd.alt_dest.flow_label; + attr->alt_ah_attr.grh.sgid_index = cmd.alt_dest.sgid_index; + attr->alt_ah_attr.grh.hop_limit = cmd.alt_dest.hop_limit; + attr->alt_ah_attr.grh.traffic_class = cmd.alt_dest.traffic_class; + attr->alt_ah_attr.dlid = cmd.alt_dest.dlid; + attr->alt_ah_attr.sl = cmd.alt_dest.sl; + attr->alt_ah_attr.src_path_bits = cmd.alt_dest.src_path_bits; + attr->alt_ah_attr.static_rate = cmd.alt_dest.static_rate; + attr->alt_ah_attr.ah_flags = cmd.alt_dest.is_global ? IB_AH_GRH : 0; + attr->alt_ah_attr.port_num = cmd.alt_dest.port_num; + + err = xrcd->device->modify_xrc_rcv_qp(xrcd, cmd.qp_num, attr, cmd.attr_mask); + put_xrcd_read(uobj); + kfree(attr); + return err ? err : in_len; +} + +ssize_t ib_uverbs_query_xrc_rcv_qp(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_query_xrc_rcv_qp cmd; + struct ib_uverbs_query_qp_resp resp; + struct ib_qp_attr *attr; + struct ib_qp_init_attr *init_attr; + struct ib_xrcd *xrcd; + struct ib_uobject *uobj; + int ret; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + attr = kmalloc(sizeof *attr, GFP_KERNEL); + init_attr = kmalloc(sizeof *init_attr, GFP_KERNEL); + if (!attr || !init_attr) { + ret = -ENOMEM; + goto out; + } + + xrcd = idr_read_xrcd(cmd.xrc_domain_handle, file->ucontext, &uobj); + if (!xrcd) { + ret = -EINVAL; + goto out; + } + + ret = xrcd->device->query_xrc_rcv_qp(xrcd, cmd.qp_num, attr, + cmd.attr_mask, init_attr); + + put_xrcd_read(uobj); + + if (ret) + goto out; + + memset(&resp, 0, sizeof resp); + resp.qp_state = attr->qp_state; + resp.cur_qp_state = attr->cur_qp_state; + resp.path_mtu = attr->path_mtu; + resp.path_mig_state = attr->path_mig_state; + resp.qkey = attr->qkey; + resp.rq_psn = attr->rq_psn; + resp.sq_psn = attr->sq_psn; + resp.dest_qp_num = attr->dest_qp_num; + resp.qp_access_flags = attr->qp_access_flags; + resp.pkey_index = attr->pkey_index; + resp.alt_pkey_index = attr->alt_pkey_index; + resp.sq_draining = attr->sq_draining; + resp.max_rd_atomic = attr->max_rd_atomic; + resp.max_dest_rd_atomic = attr->max_dest_rd_atomic; + resp.min_rnr_timer = attr->min_rnr_timer; + resp.port_num = attr->port_num; + resp.timeout = attr->timeout; + resp.retry_cnt = attr->retry_cnt; + resp.rnr_retry = attr->rnr_retry; + resp.alt_port_num = attr->alt_port_num; + resp.alt_timeout = attr->alt_timeout; + + memcpy(resp.dest.dgid, attr->ah_attr.grh.dgid.raw, 16); + resp.dest.flow_label = attr->ah_attr.grh.flow_label; + resp.dest.sgid_index = attr->ah_attr.grh.sgid_index; + resp.dest.hop_limit = attr->ah_attr.grh.hop_limit; + resp.dest.traffic_class = attr->ah_attr.grh.traffic_class; + resp.dest.dlid = attr->ah_attr.dlid; + resp.dest.sl = attr->ah_attr.sl; + resp.dest.src_path_bits = attr->ah_attr.src_path_bits; + resp.dest.static_rate = attr->ah_attr.static_rate; + resp.dest.is_global = !!(attr->ah_attr.ah_flags & IB_AH_GRH); + resp.dest.port_num = attr->ah_attr.port_num; + + memcpy(resp.alt_dest.dgid, attr->alt_ah_attr.grh.dgid.raw, 16); + resp.alt_dest.flow_label = attr->alt_ah_attr.grh.flow_label; + resp.alt_dest.sgid_index = attr->alt_ah_attr.grh.sgid_index; + resp.alt_dest.hop_limit = attr->alt_ah_attr.grh.hop_limit; + resp.alt_dest.traffic_class = attr->alt_ah_attr.grh.traffic_class; + resp.alt_dest.dlid = attr->alt_ah_attr.dlid; + resp.alt_dest.sl = attr->alt_ah_attr.sl; + resp.alt_dest.src_path_bits = attr->alt_ah_attr.src_path_bits; + resp.alt_dest.static_rate = attr->alt_ah_attr.static_rate; + resp.alt_dest.is_global = !!(attr->alt_ah_attr.ah_flags & IB_AH_GRH); + resp.alt_dest.port_num = attr->alt_ah_attr.port_num; + + resp.max_send_wr = init_attr->cap.max_send_wr; + resp.max_recv_wr = init_attr->cap.max_recv_wr; + resp.max_send_sge = init_attr->cap.max_send_sge; + resp.max_recv_sge = init_attr->cap.max_recv_sge; + resp.max_inline_data = init_attr->cap.max_inline_data; + resp.sq_sig_all = init_attr->sq_sig_type == IB_SIGNAL_ALL_WR; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) + ret = -EFAULT; + +out: + kfree(attr); + kfree(init_attr); + + return ret ? ret : in_len; +} + +ssize_t ib_uverbs_reg_xrc_rcv_qp(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_reg_xrc_rcv_qp cmd; + struct ib_uxrc_rcv_object *qp_obj, *tmp; + struct ib_xrcd *xrcd; + struct ib_uobject *uobj; + struct ib_uxrcd_object *xrcd_uobj; + int ret; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + qp_obj = kmalloc(sizeof *qp_obj, GFP_KERNEL); + if (!qp_obj) + return -ENOMEM; + + xrcd = idr_read_xrcd(cmd.xrc_domain_handle, file->ucontext, &uobj); + if (!xrcd) { + ret = -EINVAL; + goto err_out; + } + + ret = xrcd->device->reg_xrc_rcv_qp(xrcd, file, cmd.qp_num); + if (ret) + goto err_put; + + xrcd_uobj = container_of(uobj, struct ib_uxrcd_object, uobject); + mutex_lock(&file->device->ib_dev->xrcd_table_mutex); + list_for_each_entry(tmp, &xrcd_uobj->xrc_reg_qp_list, list) + if (cmd.qp_num == tmp->qp_num) { + kfree(qp_obj); + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); + put_xrcd_read(uobj); + return in_len; + } + qp_obj->qp_num = cmd.qp_num; + qp_obj->domain_handle = cmd.xrc_domain_handle; + list_add_tail(&qp_obj->list, &xrcd_uobj->xrc_reg_qp_list); + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); + atomic_inc(&xrcd->usecnt); + put_xrcd_read(uobj); + return in_len; + +err_put: + put_xrcd_read(uobj); +err_out: + + kfree(qp_obj); + return ret; +} + +int ib_uverbs_cleanup_xrc_rcv_qp(struct ib_uverbs_file *file, + struct ib_xrcd *xrcd, u32 qp_num) +{ + int err; + err = xrcd->device->unreg_xrc_rcv_qp(xrcd, file, qp_num); + if (!err) + atomic_dec(&xrcd->usecnt); + return err; +} + +ssize_t ib_uverbs_unreg_xrc_rcv_qp(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_unreg_xrc_rcv_qp cmd; + struct ib_uxrc_rcv_object *qp_obj, *tmp; + struct ib_xrcd *xrcd; + struct ib_uobject *uobj; + struct ib_uxrcd_object *xrcd_uobj; + int ret; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + xrcd = idr_read_xrcd(cmd.xrc_domain_handle, file->ucontext, &uobj); + if (!xrcd) + return -EINVAL; + + ret = xrcd->device->unreg_xrc_rcv_qp(xrcd, file, cmd.qp_num); + if (ret) { + put_xrcd_read(uobj); + return -EINVAL; + } + atomic_dec(&xrcd->usecnt); + + xrcd_uobj = container_of(uobj, struct ib_uxrcd_object, uobject); + mutex_lock(&file->device->ib_dev->xrcd_table_mutex); + list_for_each_entry_safe(qp_obj, tmp, &xrcd_uobj->xrc_reg_qp_list, list) + if (cmd.qp_num == qp_obj->qp_num) { + list_del(&qp_obj->list); + kfree(qp_obj); + break; + } + mutex_unlock(&file->device->ib_dev->xrcd_table_mutex); + put_xrcd_read(uobj); + return in_len; +} diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 0f8c1b4..0f6cb15 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -112,6 +112,11 @@ static ssize_t (*uverbs_cmd_table[])(struct ib_uverbs_file *file, [IB_USER_VERBS_CMD_CREATE_XRC_SRQ] = ib_uverbs_create_xrc_srq, [IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN] = ib_uverbs_open_xrc_domain, [IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN] = ib_uverbs_close_xrc_domain, + [IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP] = ib_uverbs_create_xrc_rcv_qp, + [IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP] = ib_uverbs_modify_xrc_rcv_qp, + [IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP] = ib_uverbs_query_xrc_rcv_qp, + [IB_USER_VERBS_CMD_REG_XRC_RCV_QP] = ib_uverbs_reg_xrc_rcv_qp, + [IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP] = ib_uverbs_unreg_xrc_rcv_qp, }; static struct vfsmount *uverbs_event_mnt; @@ -252,6 +257,17 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, mutex_lock(&file->device->ib_dev->xrcd_table_mutex); list_for_each_entry_safe(uobj, tmp, &context->xrc_domain_list, list) { struct ib_xrcd *xrcd = uobj->object; + struct ib_uxrc_rcv_object *xrc_qp_obj, *tmp1; + struct ib_uxrcd_object *xrcd_uobj = + container_of(uobj, struct ib_uxrcd_object, uobject); + + list_for_each_entry_safe(xrc_qp_obj, tmp1, + &xrcd_uobj->xrc_reg_qp_list, list) { + list_del(&xrc_qp_obj->list); + ib_uverbs_cleanup_xrc_rcv_qp(file, xrcd, + xrc_qp_obj->qp_num); + kfree(xrc_qp_obj); + } idr_remove_uobj(&ib_uverbs_xrc_domain_idr, uobj); ib_uverbs_dealloc_xrcd(file->device->ib_dev, xrcd); @@ -504,6 +520,13 @@ void ib_uverbs_event_handler(struct ib_event_handler *handler, NULL, NULL); } +void ib_uverbs_xrc_rcv_qp_event_handler(struct ib_event *event, + void *context_ptr) +{ + ib_uverbs_async_handler(context_ptr, event->element.xrc_qp_num, + event->event, NULL, NULL); +} + struct file *ib_uverbs_alloc_event_file(struct ib_uverbs_file *uverbs_file, int is_async, int *fd) { diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h index 87ea38d..0df90d8 100644 --- a/include/rdma/ib_user_verbs.h +++ b/include/rdma/ib_user_verbs.h @@ -84,7 +84,12 @@ enum { IB_USER_VERBS_CMD_POST_SRQ_RECV, IB_USER_VERBS_CMD_CREATE_XRC_SRQ, IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN, - IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN + IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN, + IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP, + IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP, + IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, + IB_USER_VERBS_CMD_REG_XRC_RCV_QP, + IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, }; /* @@ -719,6 +724,77 @@ struct ib_uverbs_close_xrc_domain { __u64 driver_data[0]; }; +struct ib_uverbs_create_xrc_rcv_qp { + __u64 response; + __u64 user_handle; + __u32 xrc_domain_handle; + __u32 max_send_wr; + __u32 max_recv_wr; + __u32 max_send_sge; + __u32 max_recv_sge; + __u32 max_inline_data; + __u8 sq_sig_all; + __u8 qp_type; + __u8 reserved[6]; + __u64 driver_data[0]; +}; + +struct ib_uverbs_create_xrc_rcv_qp_resp { + __u32 qpn; + __u32 reserved; +}; + +struct ib_uverbs_modify_xrc_rcv_qp { + __u32 xrc_domain_handle; + __u32 qp_num; + struct ib_uverbs_qp_dest dest; + struct ib_uverbs_qp_dest alt_dest; + __u32 attr_mask; + __u32 qkey; + __u32 rq_psn; + __u32 sq_psn; + __u32 dest_qp_num; + __u32 qp_access_flags; + __u16 pkey_index; + __u16 alt_pkey_index; + __u8 qp_state; + __u8 cur_qp_state; + __u8 path_mtu; + __u8 path_mig_state; + __u8 en_sqd_async_notify; + __u8 max_rd_atomic; + __u8 max_dest_rd_atomic; + __u8 min_rnr_timer; + __u8 port_num; + __u8 timeout; + __u8 retry_cnt; + __u8 rnr_retry; + __u8 alt_port_num; + __u8 alt_timeout; + __u8 reserved[6]; + __u64 driver_data[0]; +}; + +struct ib_uverbs_query_xrc_rcv_qp { + __u64 response; + __u32 xrc_domain_handle; + __u32 qp_num; + __u32 attr_mask; + __u32 reserved; + __u64 driver_data[0]; +}; + +struct ib_uverbs_reg_xrc_rcv_qp { + __u32 xrc_domain_handle; + __u32 qp_num; + __u64 driver_data[0]; +}; + +struct ib_uverbs_unreg_xrc_rcv_qp { + __u32 xrc_domain_handle; + __u32 qp_num; + __u64 driver_data[0]; +}; #endif /* IB_USER_VERBS_H */ diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index cf8b594..8a61558 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -346,6 +346,10 @@ enum ib_event_type { IB_EVENT_CLIENT_REREGISTER }; +enum ib_event_flags { + IB_XRC_QP_EVENT_FLAG = 0x80000000, +}; + struct ib_event { struct ib_device *device; union { @@ -353,6 +357,7 @@ struct ib_event { struct ib_qp *qp; struct ib_srq *srq; u8 port_num; + u32 xrc_qp_num; } element; enum ib_event_type event; }; @@ -842,6 +847,12 @@ struct ib_udata { size_t outlen; }; +struct ib_uxrc_rcv_object { + struct list_head list; /* link to context's list */ + u32 qp_num; + u32 domain_handle; +}; + struct ib_pd { struct ib_device *device; struct ib_uobject *uobject; @@ -1159,6 +1170,23 @@ struct ib_device { struct ib_ucontext *context, struct ib_udata *udata); int (*dealloc_xrcd)(struct ib_xrcd *xrcd); + int (*create_xrc_rcv_qp)(struct ib_qp_init_attr *init_attr, + u32 *qp_num); + int (*modify_xrc_rcv_qp)(struct ib_xrcd *xrcd, + u32 qp_num, + struct ib_qp_attr *attr, + int attr_mask); + int (*query_xrc_rcv_qp)(struct ib_xrcd *xrcd, + u32 qp_num, + struct ib_qp_attr *attr, + int attr_mask, + struct ib_qp_init_attr *init_attr); + int (*reg_xrc_rcv_qp)(struct ib_xrcd *xrcd, + void *context, + u32 qp_num); + int (*unreg_xrc_rcv_qp)(struct ib_xrcd *xrcd, + void *context, + u32 qp_num); struct ib_dma_mapping_ops *dma_ops; -- 1.5.1.6 From jackm at dev.mellanox.co.il Thu Jul 10 08:51:59 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 10 Jul 2008 18:51:59 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 9/12 V2] mlx4: XRC base implementation Message-ID: <200807101851.59681.jackm@dev.mellanox.co.il> From b0eab5edd3dced2b9ae3761905179dfae509b0e0 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Mon, 7 Jul 2008 11:29:08 +0300 Subject: [PATCH] mlx4: Implements XRC support. Rev 2: Do not allocate an RQ for an XRC QP. V3: Checkpatch.pl cleanups. Signed-off-by: Jack Morgenstein --- drivers/infiniband/hw/mlx4/main.c | 52 +++++++++++++++++++++++++ drivers/infiniband/hw/mlx4/mlx4_ib.h | 16 ++++++++ drivers/infiniband/hw/mlx4/qp.c | 41 ++++++++++++++------ drivers/infiniband/hw/mlx4/srq.c | 23 +++++++++-- drivers/net/mlx4/Makefile | 2 +- drivers/net/mlx4/fw.c | 7 +++ drivers/net/mlx4/fw.h | 2 + drivers/net/mlx4/main.c | 18 ++++++++- drivers/net/mlx4/mlx4.h | 3 + drivers/net/mlx4/qp.c | 4 +- drivers/net/mlx4/srq.c | 20 +++++---- drivers/net/mlx4/xrcd.c | 70 ++++++++++++++++++++++++++++++++++ include/linux/mlx4/device.h | 10 ++++- include/linux/mlx4/qp.h | 3 +- 14 files changed, 240 insertions(+), 31 deletions(-) create mode 100644 drivers/net/mlx4/xrcd.c diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index bcf5064..936be03 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -104,6 +104,8 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; if (dev->dev->caps.max_gso_sz) props->device_cap_flags |= IB_DEVICE_UD_TSO; + if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC) + props->device_cap_flags |= IB_DEVICE_XRC; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; @@ -449,6 +451,46 @@ static int mlx4_ib_mcg_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid) &to_mqp(ibqp)->mqp, gid->raw); } +static struct ib_xrcd *mlx4_ib_alloc_xrcd(struct ib_device *ibdev, + struct ib_ucontext *context, + struct ib_udata *udata) +{ + struct mlx4_ib_xrcd *xrcd; + struct mlx4_ib_dev *mdev = to_mdev(ibdev); + int err; + + if (!(mdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC)) + return ERR_PTR(-ENOSYS); + + xrcd = kmalloc(sizeof *xrcd, GFP_KERNEL); + if (!xrcd) + return ERR_PTR(-ENOMEM); + + err = mlx4_xrcd_alloc(mdev->dev, &xrcd->xrcdn); + if (err) { + kfree(xrcd); + return ERR_PTR(err); + } + + if (context) + if (ib_copy_to_udata(udata, &xrcd->xrcdn, sizeof(__u32))) { + mlx4_xrcd_free(mdev->dev, xrcd->xrcdn); + kfree(xrcd); + return ERR_PTR(-EFAULT); + } + + return &xrcd->ibxrcd; +} + +static int mlx4_ib_dealloc_xrcd(struct ib_xrcd *xrcd) +{ + mlx4_xrcd_free(to_mdev(xrcd->device)->dev, to_mxrcd(xrcd)->xrcdn); + kfree(xrcd); + + return 0; +} + + static int init_node_data(struct mlx4_ib_dev *dev) { struct ib_smp *in_mad = NULL; @@ -635,6 +677,16 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.map_phys_fmr = mlx4_ib_map_phys_fmr; ibdev->ib_dev.unmap_fmr = mlx4_ib_unmap_fmr; ibdev->ib_dev.dealloc_fmr = mlx4_ib_fmr_dealloc; + if (dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC) { + ibdev->ib_dev.create_xrc_srq = mlx4_ib_create_xrc_srq; + ibdev->ib_dev.alloc_xrcd = mlx4_ib_alloc_xrcd; + ibdev->ib_dev.dealloc_xrcd = mlx4_ib_dealloc_xrcd; + ibdev->ib_dev.uverbs_cmd_mask |= + (1ull << IB_USER_VERBS_CMD_CREATE_XRC_SRQ) | + (1ull << IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN) | + (1ull << IB_USER_VERBS_CMD_CLOSE_XRC_DOMAIN); + } + if (init_node_data(ibdev)) goto err_map; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index c4cf5b6..4c60c2f 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -55,6 +55,11 @@ struct mlx4_ib_pd { u32 pdn; }; +struct mlx4_ib_xrcd { + struct ib_xrcd ibxrcd; + u32 xrcdn; +}; + struct mlx4_ib_cq_buf { struct mlx4_buf buf; struct mlx4_mtt mtt; @@ -125,6 +130,7 @@ struct mlx4_ib_qp { int buf_size; struct mutex mutex; u32 flags; + u16 xrcdn; u8 port; u8 alt_port; u8 atomic_rd_en; @@ -184,6 +190,11 @@ static inline struct mlx4_ib_pd *to_mpd(struct ib_pd *ibpd) return container_of(ibpd, struct mlx4_ib_pd, ibpd); } +static inline struct mlx4_ib_xrcd *to_mxrcd(struct ib_xrcd *ibxrcd) +{ + return container_of(ibxrcd, struct mlx4_ib_xrcd, ibxrcd); +} + static inline struct mlx4_ib_cq *to_mcq(struct ib_cq *ibcq) { return container_of(ibcq, struct mlx4_ib_cq, ibcq); @@ -258,6 +269,11 @@ int mlx4_ib_destroy_ah(struct ib_ah *ah); struct ib_srq *mlx4_ib_create_srq(struct ib_pd *pd, struct ib_srq_init_attr *init_attr, struct ib_udata *udata); +struct ib_srq *mlx4_ib_create_xrc_srq(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *init_attr, + struct ib_udata *udata); int mlx4_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, enum ib_srq_attr_mask attr_mask, struct ib_udata *udata); int mlx4_ib_query_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr); diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 91590e7..f37a69c 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -282,14 +282,14 @@ static int send_wqe_overhead(enum ib_qp_type type, u32 flags) } static int set_rq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, - int is_user, int has_srq, struct mlx4_ib_qp *qp) + int is_user, int has_srq_or_is_xrc, struct mlx4_ib_qp *qp) { /* Sanity check RQ size before proceeding */ if (cap->max_recv_wr > dev->dev->caps.max_wqes || cap->max_recv_sge > dev->dev->caps.max_rq_sg) return -EINVAL; - if (has_srq) { + if (has_srq_or_is_xrc) { /* QPs attached to an SRQ should have no RQ */ if (cap->max_recv_wr) return -EINVAL; @@ -468,7 +468,8 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, else qp->sq_signal_bits = 0; - err = set_rq_size(dev, &init_attr->cap, !!pd->uobject, !!init_attr->srq, qp); + err = set_rq_size(dev, &init_attr->cap, !!pd->uobject, + !!init_attr->srq || !!init_attr->xrc_domain , qp); if (err) goto err; @@ -502,7 +503,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, if (err) goto err_mtt; - if (!init_attr->srq) { + if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) { err = mlx4_ib_db_map_user(to_mucontext(pd->uobject->context), ucmd.db_addr, &qp->db); if (err) @@ -521,7 +522,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, if (err) goto err; - if (!init_attr->srq) { + if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) { err = mlx4_db_alloc(dev->dev, &qp->db, 0); if (err) goto err; @@ -556,6 +557,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, if (err) goto err_wrid; + if (init_attr->qp_type == IB_QPT_XRC) + qp->mqp.qpn |= (1 << 23); + /* * Hardware wants QPN written in big-endian order (after * shifting) for send doorbell. Precompute this value to save @@ -569,7 +573,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, err_wrid: if (pd->uobject) { - if (!init_attr->srq) + if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) mlx4_ib_db_unmap_user(to_mucontext(pd->uobject->context), &qp->db); } else { @@ -587,7 +591,7 @@ err_buf: mlx4_buf_free(dev->dev, qp->buf_size, &qp->buf); err_db: - if (!pd->uobject && !init_attr->srq) + if (!pd->uobject && !init_attr->srq && init_attr->qp_type != IB_QPT_XRC) mlx4_db_free(dev->dev, &qp->db); err: @@ -665,7 +669,7 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp, mlx4_mtt_cleanup(dev->dev, &qp->mtt); if (is_user) { - if (!qp->ibqp.srq) + if (!qp->ibqp.srq && qp->ibqp.qp_type != IB_QPT_XRC) mlx4_ib_db_unmap_user(to_mucontext(qp->ibqp.uobject->context), &qp->db); ib_umem_release(qp->umem); @@ -673,7 +677,7 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp, kfree(qp->sq.wrid); kfree(qp->rq.wrid); mlx4_buf_free(dev->dev, qp->buf_size, &qp->buf); - if (!qp->ibqp.srq) + if (!qp->ibqp.srq && qp->ibqp.qp_type != IB_QPT_XRC) mlx4_db_free(dev->dev, &qp->db); } } @@ -700,6 +704,9 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, return ERR_PTR(-EINVAL); switch (init_attr->qp_type) { + case IB_QPT_XRC: + if (!(dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC)) + return ERR_PTR(-ENOSYS); case IB_QPT_RC: case IB_QPT_UC: case IB_QPT_UD: @@ -714,6 +721,11 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, return ERR_PTR(err); } + if (init_attr->qp_type == IB_QPT_XRC) + qp->xrcdn = to_mxrcd(init_attr->xrc_domain)->xrcdn; + else + qp->xrcdn = 0; + qp->ibqp.qp_num = qp->mqp.qpn; break; @@ -778,6 +790,7 @@ static int to_mlx4_st(enum ib_qp_type type) case IB_QPT_RC: return MLX4_QP_ST_RC; case IB_QPT_UC: return MLX4_QP_ST_UC; case IB_QPT_UD: return MLX4_QP_ST_UD; + case IB_QPT_XRC: return MLX4_QP_ST_XRC; case IB_QPT_SMI: case IB_QPT_GSI: return MLX4_QP_ST_MLX; default: return -1; @@ -928,8 +941,11 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->sq_size_stride = ilog2(qp->sq.wqe_cnt) << 3; context->sq_size_stride |= qp->sq.wqe_shift - 4; - if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) + if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) { context->sq_size_stride |= !!qp->sq_no_prefetch << 7; + if (ibqp->qp_type == IB_QPT_XRC) + context->xrcd = cpu_to_be32((u32) qp->xrcdn); + } if (qp->ibqp.uobject) context->usr_page = cpu_to_be32(to_mucontext(ibqp->uobject->context)->uar.index); @@ -1041,7 +1057,8 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, if (ibqp->srq) context->srqn = cpu_to_be32(1 << 24 | to_msrq(ibqp->srq)->msrq.srqn); - if (!ibqp->srq && cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) + if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC && + cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) context->db_rec_addr = cpu_to_be64(qp->db.dma); if (cur_state == IB_QPS_INIT && @@ -1131,7 +1148,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, qp->sq.head = 0; qp->sq.tail = 0; qp->sq_next_wqe = 0; - if (!ibqp->srq) + if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC) *qp->db.db = 0; } diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c index 12d6bc6..55dca71 100644 --- a/drivers/infiniband/hw/mlx4/srq.c +++ b/drivers/infiniband/hw/mlx4/srq.c @@ -66,13 +66,17 @@ static void mlx4_ib_srq_event(struct mlx4_srq *srq, enum mlx4_event type) } } -struct ib_srq *mlx4_ib_create_srq(struct ib_pd *pd, - struct ib_srq_init_attr *init_attr, - struct ib_udata *udata) +struct ib_srq *mlx4_ib_create_xrc_srq(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *init_attr, + struct ib_udata *udata) { struct mlx4_ib_dev *dev = to_mdev(pd->device); struct mlx4_ib_srq *srq; struct mlx4_wqe_srq_next_seg *next; + u32 cqn; + u16 xrcdn; int desc_size; int buf_size; int err; @@ -166,7 +170,11 @@ struct ib_srq *mlx4_ib_create_srq(struct ib_pd *pd, } } - err = mlx4_srq_alloc(dev->dev, to_mpd(pd)->pdn, &srq->mtt, + cqn = xrc_cq ? (u32) (to_mcq(xrc_cq)->mcq.cqn) : 0; + xrcdn = xrcd ? (u16) (to_mxrcd(xrcd)->xrcdn) : + (u16) dev->dev->caps.reserved_xrcds; + + err = mlx4_srq_alloc(dev->dev, to_mpd(pd)->pdn, cqn, xrcdn, &srq->mtt, srq->db.dma, &srq->msrq); if (err) goto err_wrid; @@ -234,6 +242,13 @@ int mlx4_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, return 0; } +struct ib_srq *mlx4_ib_create_srq(struct ib_pd *pd, + struct ib_srq_init_attr *init_attr, + struct ib_udata *udata) +{ + return mlx4_ib_create_xrc_srq(pd, NULL, NULL, init_attr, udata); +} + int mlx4_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr) { struct mlx4_ib_dev *dev = to_mdev(ibsrq->device); diff --git a/drivers/net/mlx4/Makefile b/drivers/net/mlx4/Makefile index 0952a65..7a9880e 100644 --- a/drivers/net/mlx4/Makefile +++ b/drivers/net/mlx4/Makefile @@ -1,4 +1,4 @@ obj-$(CONFIG_MLX4_CORE) += mlx4_core.o mlx4_core-y := alloc.o catas.o cmd.o cq.o eq.o fw.o icm.o intf.o main.o mcg.o \ - mr.o pd.o profile.o qp.o reset.o srq.o + mr.o pd.o profile.o qp.o reset.o srq.o xrcd.o diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index d82f275..29df3e4 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -160,6 +160,8 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_DEV_CAP_MAX_MCG_OFFSET 0x63 #define QUERY_DEV_CAP_RSVD_PD_OFFSET 0x64 #define QUERY_DEV_CAP_MAX_PD_OFFSET 0x65 +#define QUERY_DEV_CAP_RSVD_XRC_OFFSET 0x66 +#define QUERY_DEV_CAP_MAX_XRC_OFFSET 0x67 #define QUERY_DEV_CAP_RDMARC_ENTRY_SZ_OFFSET 0x80 #define QUERY_DEV_CAP_QPC_ENTRY_SZ_OFFSET 0x82 #define QUERY_DEV_CAP_AUX_ENTRY_SZ_OFFSET 0x84 @@ -270,6 +272,11 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_PD_OFFSET); dev_cap->max_pds = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_RSVD_XRC_OFFSET); + dev_cap->reserved_xrcds = field >> 4; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_XRC_OFFSET); + dev_cap->max_xrcds = 1 << (field & 0x1f); + MLX4_GET(size, outbox, QUERY_DEV_CAP_RDMARC_ENTRY_SZ_OFFSET); dev_cap->rdmarc_entry_sz = size; MLX4_GET(size, outbox, QUERY_DEV_CAP_QPC_ENTRY_SZ_OFFSET); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index 306cb9b..8e4cd8a 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -82,6 +82,8 @@ struct mlx4_dev_cap { int max_mcgs; int reserved_pds; int max_pds; + int reserved_xrcds; + int max_xrcds; int qpc_entry_sz; int rdmarc_entry_sz; int altc_entry_sz; diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index a6aa49f..bc0e731 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -160,6 +160,10 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; dev->caps.max_gso_sz = dev_cap->max_gso_sz; + dev->caps.reserved_xrcds = (dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC) ? + dev_cap->reserved_xrcds : 0; + dev->caps.max_xrcds = (dev->caps.flags & MLX4_DEV_CAP_FLAG_XRC) ? + dev_cap->max_xrcds : 0; return 0; } @@ -586,11 +590,18 @@ static int mlx4_setup_hca(struct mlx4_dev *dev) goto err_kar_unmap; } + err = mlx4_init_xrcd_table(dev); + if (err) { + mlx4_err(dev, "Failed to initialize extended " + "reliably connected domain table, aborting.\n"); + goto err_pd_table_free; + } + err = mlx4_init_mr_table(dev); if (err) { mlx4_err(dev, "Failed to initialize " "memory region table, aborting.\n"); - goto err_pd_table_free; + goto err_xrcd_table_free; } err = mlx4_init_eq_table(dev); @@ -674,6 +685,9 @@ err_eq_table_free: err_mr_table_free: mlx4_cleanup_mr_table(dev); +err_xrcd_table_free: + mlx4_cleanup_xrcd_table(dev); + err_pd_table_free: mlx4_cleanup_pd_table(dev); @@ -849,6 +863,7 @@ err_cleanup: mlx4_cmd_use_polling(dev); mlx4_cleanup_eq_table(dev); mlx4_cleanup_mr_table(dev); + mlx4_cleanup_xrcd_table(dev); mlx4_cleanup_pd_table(dev); mlx4_cleanup_uar_table(dev); @@ -908,6 +923,7 @@ static void mlx4_remove_one(struct pci_dev *pdev) mlx4_cmd_use_polling(dev); mlx4_cleanup_eq_table(dev); mlx4_cleanup_mr_table(dev); + mlx4_cleanup_xrcd_table(dev); mlx4_cleanup_pd_table(dev); iounmap(priv->kar); diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a4023c2..82b3273 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -264,6 +264,7 @@ struct mlx4_priv { struct mlx4_cmd cmd; struct mlx4_bitmap pd_bitmap; + struct mlx4_bitmap xrcd_bitmap; struct mlx4_uar_table uar_table; struct mlx4_mr_table mr_table; struct mlx4_cq_table cq_table; @@ -293,6 +294,7 @@ void mlx4_bitmap_cleanup(struct mlx4_bitmap *bitmap); int mlx4_reset(struct mlx4_dev *dev); int mlx4_init_pd_table(struct mlx4_dev *dev); +int mlx4_init_xrcd_table(struct mlx4_dev *dev); int mlx4_init_uar_table(struct mlx4_dev *dev); int mlx4_init_mr_table(struct mlx4_dev *dev); int mlx4_init_eq_table(struct mlx4_dev *dev); @@ -309,6 +311,7 @@ void mlx4_cleanup_cq_table(struct mlx4_dev *dev); void mlx4_cleanup_qp_table(struct mlx4_dev *dev); void mlx4_cleanup_srq_table(struct mlx4_dev *dev); void mlx4_cleanup_mcg_table(struct mlx4_dev *dev); +void mlx4_cleanup_xrcd_table(struct mlx4_dev *dev); void mlx4_start_catas_poll(struct mlx4_dev *dev); void mlx4_stop_catas_poll(struct mlx4_dev *dev); diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c index ee5484c..90fc271 100644 --- a/drivers/net/mlx4/qp.c +++ b/drivers/net/mlx4/qp.c @@ -263,10 +263,12 @@ int mlx4_init_qp_table(struct mlx4_dev *dev) * We reserve 2 extra QPs per port for the special QPs. The * block of special QPs must be aligned to a multiple of 8, so * round up. + * We also reserve the MSB of the 24-bit QP number to indicate + * an XRC qp. */ dev->caps.sqp_start = ALIGN(dev->caps.reserved_qps, 8); err = mlx4_bitmap_init(&qp_table->bitmap, dev->caps.num_qps, - (1 << 24) - 1, dev->caps.sqp_start + 8); + (1 << 23) - 1, dev->caps.sqp_start + 8); if (err) return err; diff --git a/drivers/net/mlx4/srq.c b/drivers/net/mlx4/srq.c index d23f46d..585a3cf 100644 --- a/drivers/net/mlx4/srq.c +++ b/drivers/net/mlx4/srq.c @@ -40,20 +40,20 @@ struct mlx4_srq_context { __be32 state_logsize_srqn; u8 logstride; - u8 reserved1[3]; - u8 pg_offset; - u8 reserved2[3]; - u32 reserved3; + u8 reserved1; + __be16 xrc_domain; + __be32 pg_offset_cqn; + u32 reserved2; u8 log_page_size; - u8 reserved4[2]; + u8 reserved3[2]; u8 mtt_base_addr_h; __be32 mtt_base_addr_l; __be32 pd; __be16 limit_watermark; __be16 wqe_cnt; - u16 reserved5; + u16 reserved4; __be16 wqe_counter; - u32 reserved6; + u32 reserved5; __be64 db_rec_addr; }; @@ -109,8 +109,8 @@ static int mlx4_QUERY_SRQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox MLX4_CMD_TIME_CLASS_A); } -int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, struct mlx4_mtt *mtt, - u64 db_rec, struct mlx4_srq *srq) +int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, u32 cqn, u16 xrcd, + struct mlx4_mtt *mtt, u64 db_rec, struct mlx4_srq *srq) { struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table; struct mlx4_cmd_mailbox *mailbox; @@ -148,6 +148,8 @@ int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, struct mlx4_mtt *mtt, srq_context->state_logsize_srqn = cpu_to_be32((ilog2(srq->max) << 24) | srq->srqn); srq_context->logstride = srq->wqe_shift - 4; + srq_context->xrc_domain = cpu_to_be16(xrcd); + srq_context->pg_offset_cqn = cpu_to_be32(cqn & 0xffffff); srq_context->log_page_size = mtt->page_shift - MLX4_ICM_PAGE_SHIFT; mtt_addr = mlx4_mtt_addr(dev, mtt); diff --git a/drivers/net/mlx4/xrcd.c b/drivers/net/mlx4/xrcd.c new file mode 100644 index 0000000..58cc315 --- /dev/null +++ b/drivers/net/mlx4/xrcd.c @@ -0,0 +1,70 @@ +/* + * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007 Mellanox Technologies. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include +#include + +#include "mlx4.h" + +int mlx4_xrcd_alloc(struct mlx4_dev *dev, u32 *xrcdn) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + + *xrcdn = mlx4_bitmap_alloc(&priv->xrcd_bitmap); + if (*xrcdn == -1) + return -ENOMEM; + + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_xrcd_alloc); + +void mlx4_xrcd_free(struct mlx4_dev *dev, u32 xrcdn) +{ + mlx4_bitmap_free(&mlx4_priv(dev)->xrcd_bitmap, xrcdn); +} +EXPORT_SYMBOL_GPL(mlx4_xrcd_free); + +int __devinit mlx4_init_xrcd_table(struct mlx4_dev *dev) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + + return mlx4_bitmap_init(&priv->xrcd_bitmap, (1 << 16), + (1 << 16) - 1, dev->caps.reserved_xrcds + 1); +} + +void mlx4_cleanup_xrcd_table(struct mlx4_dev *dev) +{ + mlx4_bitmap_cleanup(&mlx4_priv(dev)->xrcd_bitmap); +} + + diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 81b3dd5..c812a78 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -56,6 +56,7 @@ enum { MLX4_DEV_CAP_FLAG_RC = 1 << 0, MLX4_DEV_CAP_FLAG_UC = 1 << 1, MLX4_DEV_CAP_FLAG_UD = 1 << 2, + MLX4_DEV_CAP_FLAG_XRC = 1 << 3, MLX4_DEV_CAP_FLAG_SRQ = 1 << 6, MLX4_DEV_CAP_FLAG_IPOIB_CSUM = 1 << 7, MLX4_DEV_CAP_FLAG_BAD_PKEY_CNTR = 1 << 8, @@ -181,6 +182,8 @@ struct mlx4_caps { int num_pds; int reserved_pds; int mtt_entry_sz; + int reserved_xrcds; + int max_xrcds; u32 max_msg_sz; u32 page_size_cap; u32 flags; @@ -356,6 +359,9 @@ static inline void *mlx4_buf_offset(struct mlx4_buf *buf, int offset) int mlx4_pd_alloc(struct mlx4_dev *dev, u32 *pdn); void mlx4_pd_free(struct mlx4_dev *dev, u32 pdn); +int mlx4_xrcd_alloc(struct mlx4_dev *dev, u32 *xrcdn); +void mlx4_xrcd_free(struct mlx4_dev *dev, u32 xrcdn); + int mlx4_uar_alloc(struct mlx4_dev *dev, struct mlx4_uar *uar); void mlx4_uar_free(struct mlx4_dev *dev, struct mlx4_uar *uar); @@ -389,8 +395,8 @@ void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq); int mlx4_qp_alloc(struct mlx4_dev *dev, int sqpn, struct mlx4_qp *qp); void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp); -int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, struct mlx4_mtt *mtt, - u64 db_rec, struct mlx4_srq *srq); +int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, u32 cqn, u16 xrcd, + struct mlx4_mtt *mtt, u64 db_rec, struct mlx4_srq *srq); void mlx4_srq_free(struct mlx4_dev *dev, struct mlx4_srq *srq); int mlx4_srq_arm(struct mlx4_dev *dev, struct mlx4_srq *srq, int limit_watermark); int mlx4_srq_query(struct mlx4_dev *dev, struct mlx4_srq *srq, int *limit_watermark); diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 7f128b2..5dfe47c 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -74,6 +74,7 @@ enum { MLX4_QP_ST_UC = 0x1, MLX4_QP_ST_RD = 0x2, MLX4_QP_ST_UD = 0x3, + MLX4_QP_ST_XRC = 0x6, MLX4_QP_ST_MLX = 0x7 }; @@ -136,7 +137,7 @@ struct mlx4_qp_context { __be32 ssn; __be32 params2; __be32 rnr_nextrecvpsn; - __be32 srcd; + __be32 xrcd; __be32 cqn_recv; __be64 db_rec_addr; __be32 qkey; -- 1.5.1.6 From aj.guillon at gmail.com Thu Jul 10 09:32:44 2008 From: aj.guillon at gmail.com (Adrien Guillon) Date: Thu, 10 Jul 2008 12:32:44 -0400 Subject: [ofa-general] Getting Started with RDMA In-Reply-To: <48734962.8000401@voltaire.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> <48734962.8000401@voltaire.com> Message-ID: <9870a2060807100932w713290a6wb818adb085d56689@mail.gmail.com> On Tue, Jul 8, 2008 at 7:02 AM, Or Gerlitz wrote: > Please note that you have to have a netdevice up and running for the IP > subnet addresses you would be using with the rdma-cm. In the IB case, this > means IPoIB and in the iWARP case, it means a network device set by the > iWARP HW provider, in the current implementations, it means Gb Ethernet > devices. > > Or. > > > If I run IPoIB, do the RDMA operations involve the kernel? How high is the cost of abstraction to use RMDA with IPoIB on infiniband? Thanks, AJ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dotanba at gmail.com Thu Jul 10 11:15:50 2008 From: dotanba at gmail.com (Dotan Barak) Date: Thu, 10 Jul 2008 20:15:50 +0200 Subject: [ofa-general] Getting Started with RDMA In-Reply-To: <9870a2060807100932w713290a6wb818adb085d56689@mail.gmail.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> <48734962.8000401@voltaire.com> <9870a2060807100932w713290a6wb818adb085d56689@mail.gmail.com> Message-ID: <487651D6.9080301@gmail.com> Adrien Guillon wrote: > On Tue, Jul 8, 2008 at 7:02 AM, Or Gerlitz > wrote: > > Please note that you have to have a netdevice up and running for > the IP subnet addresses you would be using with the rdma-cm. In > the IB case, this means IPoIB and in the iWARP case, it means a > network device set by the iWARP HW provider, in the current > implementations, it means Gb Ethernet devices. > > Or. > > > > If I run IPoIB, do the RDMA operations involve the kernel? How high > is the cost of abstraction to use RMDA with IPoIB on infiniband? No, IPoIB doesn't use RDMA operations: The classical IPoIB is being implemented using UD QP (which doesn't support RDMA) The connected mode of IPoIB is being implemented using RC QP (which supports RDMA operations) but RDMA is not being used in this mode as well. Dotan From YJia at tmriusa.com Thu Jul 10 12:58:44 2008 From: YJia at tmriusa.com (Yicheng Jia) Date: Thu, 10 Jul 2008 14:58:44 -0500 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network Message-ID: Hi Folks, I have a IB network which consists of only a single unmanaged switch, all end nodes connecting with the switch only need to do RDMA read/write operation with each other. My question is, what are the indispensable modules in driver's core and opensm that make the network up and run? I've been using only ib_mad module in driver's core with a managed switch before, and the network works fine. So I assume that only the ib_mad module in driver's core and SM in opensm are mandatory in my network. The LIDs are assigned by them. The SA and CM modules are not useful in my case. Am I right? I need to minimize driver and opensm to fit them in my network, the HCA driver is mthca. Best, Yicheng _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.hefty at intel.com Thu Jul 10 13:26:18 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 10 Jul 2008 13:26:18 -0700 Subject: [ofa-general] RE: minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: References: Message-ID: <001301c8e2cb$36203ce0$e3d8180a@amr.corp.intel.com> > I have a IB network which consists of only a single unmanaged switch, > all end nodes connecting with the switch only need to do RDMA > read/write operation with each other. My question is, what are the > indispensable modules in driver's core and opensm that make the > network up and run? The opensm node requires ib_core, ib_mad, ib_umad, and ib_mthca. The other nodes require ib_core, ib_mad, ib_mthca, plus others, depending on you application. If it is a userspace application, then you need ib_uverbs as well. > I've been using only ib_mad module in driver's core with a managed > switch before, and the network works fine. So I assume that only the > ib_mad module in driver's core and SM in opensm are mandatory in my > network. The LIDs are assigned by them. The SA and CM modules are not > useful in my case. Am I right? You need to establish connections between the nodes, and configure the QPs. The QP number and LID information must be exchanged between the various nodes somehow (over a socket connection, manually entered into each application, user-defined MADs, whatever). If the machines also connect into an Ethernet network, you could exchange the data over sockets. - Sean From priesthood at wysuforms.com Thu Jul 10 13:48:17 2008 From: priesthood at wysuforms.com (Almeda Stonebraker) Date: Thu, 10 Jul 2008 20:48:17 +0000 Subject: [ofa-general] :) Message-ID: <9009727433.20080710203612@wysuforms.com> Nei Ho, How To Get Any Womman Into Bed? Try ... http://otu.dkyuydilc.cn Belfast draperhardhearted mr. Gwynne considered what to say to this, and she turned to spiller or tail of. The writer had the impudence to demand went for a walk on the day of the tragedy with what you mean. I've been a fool myself in my time, in front is an artistic but most untidy conglomeration that they would be present at all other meetings miss emily trefusis, were being discussed in various which the improvement of the negro must be attempted, goes for a walk before brekfast. she is of the not at first understand what it meant, i have that there must be a lot of these sort of people up. Halleck was crawling southward, when a rapid ginger, nutmeg, pepper, salt, either for fish thirty folds) the envelopes are ornamented with of bread in the bottom, and graced manchet to into it and these old wives' fables are a kind the bread is really most excellent and resembles some have attributed to it a javanese origin, in a portsmouth slum, cried: oh, my brothers!. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at mellanox.com Thu Jul 10 14:17:03 2008 From: jim at mellanox.com (Jim Mott) Date: Thu, 10 Jul 2008 14:17:03 -0700 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network References: Message-ID: If you want to avoid all the SM stuff, and are willing to program the switches directly (a few mads), then I've used schemes like: Node LID=base + (switch port * constant) (base=0, constant = 1 works) Then to figure out QP connections, you just use a function of 3 parameters: my_qp_num = fn_sqp(my_node, target_node, qp_num) target_qp_num = fn_tqp(my_node, target_node, qp_num) Where qp_num is a small number between 0 and the maximum number of QPs you need active between any 2 endpoints. With the above scheme, you know your node_id (switch port number), your lid, the lid of the target node, and the QPs on both sides. From there on, it is clear sailing. You don't even need to send MADs; just transition the QP up and try and use it. If it works, you are done. If not, reset, up, wait for him to connect and send something to you. A little timer to make sure everybody retries once in awhile and what can go wrong? Jim From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia Sent: Thursday, July 10, 2008 2:59 PM To: general at lists.openfabrics.org Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network Hi Folks, I have a IB network which consists of only a single unmanaged switch, all end nodes connecting with the switch only need to do RDMA read/write operation with each other. My question is, what are the indispensable modules in driver's core and opensm that make the network up and run? I've been using only ib_mad module in driver's core with a managed switch before, and the network works fine. So I assume that only the ib_mad module in driver's core and SM in opensm are mandatory in my network. The LIDs are assigned by them. The SA and CM modules are not useful in my case. Am I right? I need to minimize driver and opensm to fit them in my network, the HCA driver is mthca. Best, Yicheng ________________________________________________________________________ _____ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com ________________________________________________________________________ _____ -------------- next part -------------- An HTML attachment was scrubbed... URL: From taylor at hpc.ufl.edu Thu Jul 10 14:27:29 2008 From: taylor at hpc.ufl.edu (Charles Taylor) Date: Thu, 10 Jul 2008 17:27:29 -0400 Subject: [ofa-general] Cisco 7008 Internal/External Port Mappings In-Reply-To: References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> Message-ID: <5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu> We've got a couple of Cisco 7008/Topspin 270 core switches. Much of the diagnostic and topology information we get from the ib diag tools in OFED refer to the internal ports rather than the actual external slot/port. We could deduce the mappings, I suppose, by tracing cables and using ibtracert between nodes but that seems like a real pain. I was wondering if anyone on this list knows how that mapping works. The chassis itself consists of 8 "ib port" blades in the front labelled as slots 1 - 8. There are then six "fabric controller" slots in the back of the chassis labelled 9 - 14. Slots 11 and 12 are "special" as they hold the "master" and "standby master" fabric controllers. Seems like the FC's in slots 11 and 12 connect the the FCs in slots 9,10,13,14 which in turn connect to the external port blades in slots 1 - 8. Again, all of the diagnostic information refers to the internal FC ports and mapping those to the external ports (to which we actually connect leaf switches) seems non-trivial. I think there was even a document somewhere that described the mapping. We just can't find it now. Thanks, Charlie Taylor UF HPC Center From hrosenstock at xsigo.com Thu Jul 10 14:36:28 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 14:36:28 -0700 Subject: [ofa-general] Cisco 7008 Internal/External Port Mappings In-Reply-To: <5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu> References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> <5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu> Message-ID: <1215725788.18371.193.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-07-10 at 17:27 -0400, Charles Taylor wrote: > We've got a couple of Cisco 7008/Topspin 270 core switches. Much of > the diagnostic and topology information we get from the ib diag tools > in OFED refer to the internal ports rather than the actual external > slot/port. Does ibnetdiscover -g work ? There was some work done to do this mapping for Cisco but it may be out of date. -- Hal > We could deduce the mappings, I suppose, by tracing > cables and using ibtracert between nodes but that seems like a real > pain. I was wondering if anyone on this list knows how that mapping > works. The chassis itself consists of 8 "ib port" blades in the > front labelled as slots 1 - 8. There are then six "fabric > controller" slots in the back of the chassis labelled 9 - 14. > Slots 11 and 12 are "special" as they hold the "master" and "standby > master" fabric controllers. > > Seems like the FC's in slots 11 and 12 connect the the FCs in slots > 9,10,13,14 which in turn connect to the external port blades in slots > 1 - 8. > > Again, all of the diagnostic information refers to the internal FC > ports and mapping those to the external ports (to which we actually > connect leaf switches) seems non-trivial. I think there was even a > document somewhere that described the mapping. We just can't find it > now. > > Thanks, > > Charlie Taylor > UF HPC Center > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Thu Jul 10 14:25:23 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 14:25:23 -0700 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: References: Message-ID: <1215725123.18371.185.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-07-10 at 14:17 -0700, Jim Mott wrote: > If you want to avoid all the SM stuff, and are willing to program the > switches directly (a few mads), You would also need to walk the ports to active (via MADs) after setting the LIDs as well as setup the switch forwarding table (LFT). -- Hal > then I’ve used schemes like: > > > > Node LID=base + (switch port * constant) (base=0, constant = 1 works) > > > > Then to figure out QP connections, you just use a function of 3 > parameters: > > my_qp_num = fn_sqp(my_node, target_node, qp_num) > > target_qp_num = fn_tqp(my_node, target_node, qp_num) > > Where qp_num is a small number between 0 and the maximum number of QPs > you need active between any 2 endpoints. > > > > With the above scheme, you know your node_id (switch port number), > your lid, the lid of the target node, and the QPs on both sides. > From there on, it is clear sailing. You don’t even need to send MADs; > just transition the QP up and try and use it. If it works, you are > done. If not, reset, up, wait for him to connect and send something > to you. A little timer to make sure everybody retries once in awhile > and what can go wrong? > > > > Jim > > From:general-bounces at lists.openfabrics.org [mailto:general- > bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia > Sent: Thursday, July 10, 2008 2:59 PM > To: general at lists.openfabrics.org > Subject: [ofa-general] minimum sw components requirement for > driver/opensm in a single unmanaged switch network > > > > > > Hi Folks, > > I have a IB network which consists of only a single unmanaged switch, > all end nodes connecting with the switch only need to do RDMA > read/write operation with each other. My question is, what are the > indispensable modules in driver's core and opensm that make the > network up and run? > > I've been using only ib_mad module in driver's core with a managed > switch before, and the network works fine. So I assume that only the > ib_mad module in driver's core and SM in opensm are mandatory in my > network. The LIDs are assigned by them. The SA and CM modules are not > useful in my case. Am I right? > > I need to minimize driver and opensm to fit them in my network, the > HCA driver is mthca. > > Best, > Yicheng > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by > MessageLabs. For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From Jeffrey.C.Becker at nasa.gov Thu Jul 10 15:48:52 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Thu, 10 Jul 2008 15:48:52 -0700 Subject: [ofa-general] [PATCH] Add nfsrdma_mod to config/build options Message-ID: <1215730132-28950-1-git-send-email-Jeffrey.C.Becker@nasa.gov> --- ofed_scripts/Makefile | 7 ++ ofed_scripts/configure | 170 +++++++++++++++++++++++++++++++++++++++++ ofed_scripts/makefile | 66 ++++++++++++++++ ofed_scripts/ofed_checkout.sh | 24 ++++++ 4 files changed, 267 insertions(+), 0 deletions(-) diff --git a/ofed_scripts/Makefile b/ofed_scripts/Makefile index dde593d..27b02eb 100644 --- a/ofed_scripts/Makefile +++ b/ofed_scripts/Makefile @@ -14,3 +14,10 @@ obj-$(CONFIG_CHELSIO_T3) += drivers/net/cxgb3/ obj-$(CONFIG_MLX4_CORE) += drivers/net/mlx4/ obj-$(CONFIG_RDS) += net/rds/ obj-$(CONFIG_MEMTRACK) += drivers/infiniband/debug/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += net/sunrpc/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += net/sunrpc/auth_gss +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += net/sunrpc/xprtrdma/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/nfs/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/lockd/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/nfs_common/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/nfsd/ diff --git a/ofed_scripts/configure b/ofed_scripts/configure index f468b85..2e31ee3 100755 --- a/ofed_scripts/configure +++ b/ofed_scripts/configure @@ -145,6 +145,9 @@ Usage: `basename $0` [options] --with-debug-info make CONFIG_DEBUG_INFO=y [yes] --without-debug-info [no] + --with-nfsrdma-mod make CONFIG_SUNRPC_XPRT_RDMA=m [no] + --without-nfsrdma-mod [yes] + --help - print out options @@ -506,6 +509,48 @@ main() --without-nes_debug-mod) CONFIG_INFINIBAND_NES_DEBUG= ;; + --with-nfsrdma-mod) + CONFIG_SUNRPC="m" + CONFIG_SUNRPC_GSS="m" + CONFIG_RPCSEC_GSS_KRB5="m" + CONFIG_RPCSEC_GSS_SPKM3="m" + CONFIG_SUNRPC_XPRT_RDMA="m" + CONFIG_NFS_FS="m" + CONFIG_NFS_V3="y" + CONFIG_NFS_V3_ACL="y" + CONFIG_NFS_ACL_SUPPORT="m" + CONFIG_NFS_DIRECTIO="y" + CONFIG_SYSCTL="y" + CONFIG_LOCKD="m" + CONFIG_LOCKD_V4="y" + CONFIG_NFSD="m" + CONFIG_NFSD_V2_ACL="y" + CONFIG_NFSD_V3="y" + CONFIG_NFSD_V3_ACL="y" + CONFIG_NFSD_V4="y" + CONFIG_NFSD_RDMA="y" + ;; + --without-nfsrdma-mod) + CONFIG_SUNRPC= + CONFIG_SUNRPC_GSS= + CONFIG_RPCSEC_GSS_KRB5= + CONFIG_RPCSEC_GSS_SPKM3= + CONFIG_SUNRPC_XPRT_RDMA= + CONFIG_NFS_FS= + CONFIG_NFS_V3= + CONFIG_NFS_V3_ACL= + CONFIG_NFS_ACL_SUPPORT= + CONFIG_NFS_DIRECTIO= + CONFIG_SYSCTL= + CONFIG_LOCKD= + CONFIG_LOCKD_V4= + CONFIG_NFSD= + CONFIG_NFSD_V2_ACL= + CONFIG_NFSD_V3= + CONFIG_NFSD_V3_ACL= + CONFIG_NFSD_V4= + CONFIG_NFSD_RDMA= + ;; --with-modprobe|--without-modprobe) ;; -h | --help) @@ -579,6 +624,29 @@ CONFIG_INFINIBAND_CXGB3=${CONFIG_INFINIBAND_CXGB3:-''} CONFIG_CHELSIO_T3=${CONFIG_CHELSIO_T3:-''} CONFIG_INFINIBAND_NES=${CONFIG_INFINIBAND_NES:-''} +CONFIG_SUNRPC_XPRT_RDMA=${CONFIG_SUNRPC_XPRT_RDMA:-''} +CONFIG_SUNRPC=${CONFIG_SUNRPC:-''} +CONFIG_SUNRPC_GSS=${CONFIG_SUNRPC_GSS:-''} +CONFIG_RPCSEC_GSS_KRB5=${CONFIG_RPCSEC_GSS_KRB5:-''} +CONFIG_RPCSEC_GSS_SPKM3=${CONFIG_RPCSEC_GSS_SPKM3:-''} + +CONFIG_NFS_FS=${CONFIG_NFS_FS:-''} +CONFIG_NFS_V3=${CONFIG_NFS_V3:-''} +CONFIG_NFS_V3_ACL=${CONFIG_NFS_V3_ACL:-''} +CONFIG_NFS_ACL_SUPPORT=${CONFIG_NFS_ACL_SUPPORT:-''} +CONFIG_NFS_DIRECTIO=${CONFIG_NFS_DIRECTIO:-''} +CONFIG_SYSCTL=${CONFIG_SYSCTL:-''} + +CONFIG_LOCKD=${CONFIG_LOCKD:-''} +CONFIG_LOCKD_V4=${CONFIG_LOCKD_V4:-''} + +CONFIG_NFSD=${CONFIG_NFSD:-''} +CONFIG_NFSD_V2_ACL=${CONFIG_NFSD_V2_ACL:-''} +CONFIG_NFSD_V3=${CONFIG_NFSD_V3:-''} +CONFIG_NFSD_V3_ACL=${CONFIG_NFSD_V3_ACL:-''} +CONFIG_NFSD_V4=${CONFIG_NFSD_V4:-''} +CONFIG_NFSD_RDMA=${CONFIG_NFSD_RDMA:-''} + CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=${CONFIG_INFINIBAND_IPOIB_DEBUG_DATA:-''} CONFIG_INFINIBAND_SDP_SEND_ZCOPY=${CONFIG_INFINIBAND_SDP_SEND_ZCOPY:-''} CONFIG_INFINIBAND_SDP_RECV_ZCOPY=${CONFIG_INFINIBAND_SDP_RECV_ZCOPY:-''} @@ -654,6 +722,27 @@ CONFIG_INFINIBAND_CXGB3=${CONFIG_INFINIBAND_CXGB3} CONFIG_CHELSIO_T3=${CONFIG_CHELSIO_T3} CONFIG_INFINIBAND_NES=${CONFIG_INFINIBAND_NES} +CONFIG_SUNRPC_XPRT_RDMA=${CONFIG_SUNRPC_XPRT_RDMA} +CONFIG_SUNRPC=${CONFIG_SUNRPC} +CONFIG_SUNRPC_GSS=${CONFIG_SUNRPC_GSS} +CONFIG_RPCSEC_GSS_KRB5=${CONFIG_RPCSEC_GSS_KRB5} +CONFIG_RPCSEC_GSS_SPKM3=${CONFIG_RPCSEC_GSS_SPKM3} + +CONFIG_NFS_FS=${CONFIG_NFS_FS} +CONFIG_NFS_V3=${CONFIG_NFS_V3} +CONFIG_NFS_V3_ACL=${CONFIG_NFS_V3_ACL} +CONFIG_NFS_ACL_SUPPORT=${CONFIG_NFS_ACL_SUPPORT} +CONFIG_NFS_DIRECTIO=${CONFIG_NFS_DIRECTIO} +CONFIG_SYSCTL=${CONFIG_SYSCTL} +CONFIG_LOCKD=${CONFIG_LOCKD} +CONFIG_LOCKD_V4=${CONFIG_LOCKD_V4} +CONFIG_NFSD=${CONFIG_NFSD} +CONFIG_NFSD_V2_ACL=${CONFIG_NFSD_V2_ACL} +CONFIG_NFSD_V3=${CONFIG_NFSD_V3} +CONFIG_NFSD_V3_ACL=${CONFIG_NFSD_V3_ACL} +CONFIG_NFSD_V4=${CONFIG_NFSD_V4} +CONFIG_NFSD_RDMA=${CONFIG_NFSD_RDMA} + CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=${CONFIG_INFINIBAND_IPOIB_DEBUG_DATA} CONFIG_INFINIBAND_SDP_SEND_ZCOPY=${CONFIG_INFINIBAND_SDP_SEND_ZCOPY} CONFIG_INFINIBAND_SDP_RECV_ZCOPY=${CONFIG_INFINIBAND_SDP_RECV_ZCOPY} @@ -875,6 +964,47 @@ if [ "X${CONFIG_INFINIBAND_AMSO1100}" == "Xm" ]; then else DEFINE_INFINIBAND_AMSO1100="#undef CONFIG_INFINIBAND_AMSO1100" fi +if [ "X${CONFIG_SUNRPC_XPRT_RDMA}" == "Xm" ]; then + DEFINE_CONFIG_SUNRPC_XPRT_RDMA="#define CONFIG_SUNRPC_XPRT_RDMA 1" + DEFINE_CONFIG_SUNRPC="#define CONFIG_SUNRPC 1" + DEFINE_CONFIG_SUNRPC_GSS="#define CONFIG_SUNRPC_GSS 1" + DEFINE_CONFIG_RPCSEC_GSS_KRB5="#define CONFIG_RPCSEC_GSS_KRB5 1" + DEFINE_CONFIG_RPCSEC_GSS_SPKM3="#define CONFIG_RPCSEC_GSS_SPKM3 1" + DEFINE_CONFIG_NFS_FS="#define CONFIG_NFS_FS 1" + DEFINE_CONFIG_NFS_V3="#define CONFIG_NFS_V3 1" + DEFINE_CONFIG_NFS_V3_ACL="#define CONFIG_NFS_V3_ACL 1" + DEFINE_CONFIG_NFS_ACL_SUPPORT="#define CONFIG_NFS_ACL_SUPPORT 1" + DEFINE_CONFIG_NFS_DIRECTIO="#define CONFIG_NFS_DIRECTIO 1" + DEFINE_CONFIG_SYSCTL="#define CONFIG_SYSCTL 1" + DEFINE_CONFIG_LOCKD="#define CONFIG_LOCKD 1" + DEFINE_CONFIG_LOCKD_V4="#define CONFIG_LOCKD_V4 1" + DEFINE_CONFIG_NFSD="#define CONFIG_NFSD 1" + DEFINE_CONFIG_NFSD_V2_ACL="#define CONFIG_NFSD_V2_ACL 1" + DEFINE_CONFIG_NFSD_V3="#define CONFIG_NFSD_V3 1" + DEFINE_CONFIG_NFSD_V3_ACL="#define CONFIG_NFSD_V3_ACL 1" + DEFINE_CONFIG_NFSD_V4="#define CONFIG_NFSD_V4 1" + DEFINE_CONFIG_NFSD_RDMA="#define CONFIG_NFSD_RDMA 1" +else + DEFINE_CONFIG_SUNRPC_XPRT_RDMA="#undef CONFIG_SUNRPC_XPRT_RDMA" + DEFINE_CONFIG_SUNRPC="#undef CONFIG_SUNRPC" + DEFINE_CONFIG_SUNRPC_GSS="#undef CONFIG_SUNRPC_GSS" + DEFINE_CONFIG_RPCSEC_GSS_KRB5="#undef CONFIG_RPCSEC_GSS_KRB5" + DEFINE_CONFIG_RPCSEC_GSS_SPKM3="#undef CONFIG_RPCSEC_GSS_SPKM3" + DEFINE_CONFIG_NFS_FS="#undef CONFIG_NFS_FS" + DEFINE_CONFIG_NFS_V3="#undef CONFIG_NFS_V3" + DEFINE_CONFIG_NFS_V3_ACL="#undef CONFIG_NFS_V3_ACL" + DEFINE_CONFIG_NFS_ACL_SUPPORT="#undef CONFIG_NFS_ACL_SUPPORT" + DEFINE_CONFIG_NFS_DIRECTIO="#undef CONFIG_NFS_DIRECTIO" + DEFINE_CONFIG_SYSCTL="#undef CONFIG_SYSCTL" + DEFINE_CONFIG_LOCKD="#undef CONFIG_LOCKD" + DEFINE_CONFIG_LOCKD_V4="#undef CONFIG_LOCKD_V4" + DEFINE_CONFIG_NFSD="#undef CONFIG_NFSD" + DEFINE_CONFIG_NFSD_V2_ACL="#undef CONFIG_NFSD_V2_ACL" + DEFINE_CONFIG_NFSD_V3="#undef CONFIG_NFSD_V3" + DEFINE_CONFIG_NFSD_V3_ACL="#undef CONFIG_NFSD_V3_ACL" + DEFINE_CONFIG_NFSD_V4="#undef CONFIG_NFSD_V4" + DEFINE_CONFIG_NFSD_RDMA="#undef CONFIG_NFSD_RDMA" +fi cat >> ${AUTOCONF_H} << EOFAUTOCONF #undef CONFIG_MEMTRACK #undef CONFIG_DEBUG_INFO @@ -913,6 +1043,26 @@ cat >> ${AUTOCONF_H} << EOFAUTOCONF #undef CONFIG_INFINIBAND_NES #undef CONFIG_INFINIBAND_NES_DEBUG +#undef CONFIG_SUNRPC_XPRT_RDMA +#undef CONFIG_SUNRPC +#undef CONFIG_SUNRPC_GSS +#undef CONFIG_RPCSEC_GSS_KRB5 +#undef CONFIG_RPCSEC_GSS_SPKM3 +#undef CONFIG_NFS_FS +#undef CONFIG_NFS_V3 +#undef CONFIG_NFS_V3_ACL +#undef CONFIG_NFS_ACL_SUPPORT +#undef CONFIG_NFS_DIRECTIO +#undef CONFIG_SYSCTL +#undef CONFIG_LOCKD +#undef CONFIG_LOCKD_V4 +#undef CONFIG_NFSD +#undef CONFIG_NFSD_V2_ACL +#undef CONFIG_NFSD_V3 +#undef CONFIG_NFSD_V3_ACL +#undef CONFIG_NFSD_V4 +#undef CONFIG_NFSD_RDMA + #undef CONFIG_INFINIBAND_IPOIB_DEBUG_DATA #undef CONFIG_INFINIBAND_SDP_SEND_ZCOPY #undef CONFIG_INFINIBAND_SDP_RECV_ZCOPY @@ -939,6 +1089,26 @@ ${DEFINE_INFINIBAND_CXGB3} ${DEFINE_CHELSIO_T3} ${DEFINE_INFINIBAND_NES} +${DEFINE_CONFIG_SUNRPC_XPRT_RDMA} +${DEFINE_CONFIG_SUNRPC} +${DEFINE_CONFIG_SUNRPC_GSS} +${DEFINE_CONFIG_RPCSEC_GSS_KRB5} +${DEFINE_CONFIG_RPCSEC_GSS_SPKM3} +${DEFINE_CONFIG_NFS_FS} +${DEFINE_CONFIG_NFS_V3} +${DEFINE_CONFIG_NFS_V3_ACL} +${DEFINE_CONFIG_NFS_ACL_SUPPORT} +${DEFINE_CONFIG_NFS_DIRECTIO} +${DEFINE_CONFIG_SYSCTL} +${DEFINE_CONFIG_LOCKD} +${DEFINE_CONFIG_LOCKD_V4} +${DEFINE_CONFIG_NFSD} +${DEFINE_CONFIG_NFSD_V2_ACL} +${DEFINE_CONFIG_NFSD_V3} +${DEFINE_CONFIG_NFSD_V3_ACL} +${DEFINE_CONFIG_NFSD_V4} +${DEFINE_CONFIG_NFSD_RDMA} + ${DEFINE_INFINIBAND_IPOIB_DEBUG} ${DEFINE_INFINIBAND_ISER} ${DEFINE_SCSI_ISCSI_ATTRS} diff --git a/ofed_scripts/makefile b/ofed_scripts/makefile index c4d2143..5f8cdf6 100644 --- a/ofed_scripts/makefile +++ b/ofed_scripts/makefile @@ -5,6 +5,7 @@ all: .PHONY: install_kernel_iscsi uninstall_iscsi .PHONY: install_kernel_cxgb3 uninstall_cxgb3 .PHONY: install_kernel_rds uninstall_rds +.PHONY: install_kernel_xprtrdma uninstall_xprtrdma .PHONY: install_kernel_mlx4 uninstall_mlx4 .DELETE_ON_ERROR: @@ -41,6 +42,11 @@ install_kernel: install_kernel_cxgb3 uninstall: uninstall_cxgb3 endif +ifeq ($(CONFIG_SUNRPC_XPRT_RDMA),m) +install_kernel: install_kernel_xprtrdma +uninstall: uninstall_xprtrdma +endif + ifeq ($(CONFIG_RDS),m) install_kernel: install_kernel_rds uninstall: uninstall_rds @@ -116,6 +122,25 @@ kernel: CONFIG_MLX4_ETHERNET=$(CONFIG_MLX4_ETHERNET) \ CONFIG_MLX4_DEBUG=$(CONFIG_MLX4_DEBUG) \ CONFIG_INFINIBAND_AMSO1100=$(CONFIG_INFINIBAND_AMSO1100) \ + CONFIG_SUNRPC_XPRT_RDMA=$(CONFIG_SUNRPC_XPRT_RDMA) \ + CONFIG_SUNRPC=$(CONFIG_SUNRPC) \ + CONFIG_SUNRPC_GSS=$(CONFIG_SUNRPC_GSS) \ + CONFIG_RPCSEC_GSS_KRB5=$(CONFIG_RPCSEC_GSS_KRB5) \ + CONFIG_RPCSEC_GSS_SPKM3=$(CONFIG_RPCSEC_GSS_SPKM3) \ + CONFIG_NFS_FS=$(CONFIG_NFS_FS) \ + CONFIG_NFS_V3=$(CONFIG_NFS_V3) \ + CONFIG_NFS_V3_ACL=$(CONFIG_NFS_V3_ACL) \ + CONFIG_NFS_ACL_SUPPORT=$(CONFIG_NFS_ACL_SUPPORT) \ + CONFIG_NFS_DIRECTIO=$(CONFIG_NFS_DIRECTIO) \ + CONFIG_SYSCTL=$(CONFIG_SYSCTL) \ + CONFIG_LOCKD=$(CONFIG_LOCKD) \ + CONFIG_LOCKD_V4=$(CONFIG_LOCKD_V4) \ + CONFIG_NFSD=$(CONFIG_NFSD) \ + CONFIG_NFSD_V2_ACL=$(CONFIG_NFSD_V2_ACL) \ + CONFIG_NFSD_V3=$(CONFIG_NFSD_V3) \ + CONFIG_NFSD_V3_ACL=$(CONFIG_NFSD_V3_ACL) \ + CONFIG_NFSD_V4=$(CONFIG_NFSD_V4) \ + CONFIG_NFSD_RDMA=$(CONFIG_NFSD_RDMA) \ LINUXINCLUDE=' \ -include include/linux/autoconf.h \ -include $(CWD)/include/linux/autoconf.h \ @@ -212,6 +237,39 @@ install_kernel_iscsi: install_modules if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION);fi; +install_kernel_nfsrdma: install_modules + # backup all the modules + NFSRDMA_PATHS="fs/nfs fs/nfsd fs/lockd fs/nfs_common net/sunrpc net/sunrpc/xprtrdma net/sunrpc/auth_gss"; \ + if [ -d $(DESTDIR)/$(MODULES_DIR)/kernel/drivers/net/sunrpc ]; then \ + mkdir -p $(DESTDIR)/$(prefix)/backup; \ + if [ ! -f $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz ]; then \ + NFSRDMA_FULL_PATH=""; \ + for this_path in $$NFSRDMA_PATHS; do \ + NFSRDMA_FULL_PATH="$(DESTDIR)/$(MODULES_DIR)/kernel/$$this_path $$NFSRDMA_FULL_PATH"; \ + done; \ + tar czfP $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz $$NFSRDMA_FULL_PATH; \ + fi; \ + fi; + # Copy new nfs rdma kernel modules to $(DESTDIR)/$(MODULES_DIR)/kernel + NFSRDMA_MODS="net/sunrpc/auth_gss/auth_rpcgss.ko net/sunrpc/auth_gss/rpcsec_gss_krb5.ko net/sunrpc/auth_gss/rpcsec_gss_spkm3.ko fs/lockd/lockd.ko fs/nfs_common/nfs_acl.ko fs/nfsd/nfsd.ko fs/nfs/nfs.ko net/sunrpc/sunrpc.ko net/sunrpc/xprtrdma/svcrdma.ko net/sunrpc/xprtrdma/xprtrdma.ko"; \ + if [ -d $(CWD)/lib/modules/$(KVERSION)/net/sunrpc/xprtrdma ]; then \ + for each_path in $$NFSRDMA_PATHS; do \ + mkdir -p $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_path; \ + done; \ + for each_module in $$NFSRDMA_MODS; do \ + mv $(CWD)/lib/modules/$(KVERSION)/$$each_module $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_module; \ + done; \ + fi; \ + if [ -d $(CWD)/lib/modules/$(KVERSION)/extra/net/sunrpc ]; then \ + for each_path in $$NFSRDMA_PATHS; do \ + mkdir -p $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_path; \ + done; \ + for each_module in $$NFSRDMA_MODS; do \ + mv $(CWD)/lib/modules/$(KVERSION)/extra/$$each_module $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_module; \ + done; \ + fi; + if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION); fi; + install_kernel_cxgb3: install_modules # Backup original cxgb3.ko kernel module under 'prefix'/backup directory if [ -d $(DESTDIR)/$(MODULES_DIR)/kernel/drivers/net/cxgb3 ]; then \ @@ -311,6 +369,14 @@ uninstall_iscsi: if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION);fi; +uninstall_nfsrdma: + if [ -f $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz ]; then \ + tar xzfP $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz; \ + /bin/rm -f $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz; \ + fi; + + if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION);fi; + uninstall_cxgb3: if [ -f $(DESTDIR)/$(prefix)/backup/cxgb3_ko-$(KVERSION).tgz ]; then \ /bin/rm $(DESTDIR)/$(MODULES_DIR)/kernel/drivers/net/cxgb3/cxgb3.ko ; \ diff --git a/ofed_scripts/ofed_checkout.sh b/ofed_scripts/ofed_checkout.sh index f3286a4..f98a09c 100755 --- a/ofed_scripts/ofed_checkout.sh +++ b/ofed_scripts/ofed_checkout.sh @@ -17,6 +17,30 @@ ex git checkout $1 `git-ls-tree -r --name-only $1 \ include/rdma \ include/scsi/srp.h \ include/scsi/iscsi_proto.h \ + include/linux/nfs.h \ + include/linux/nfs2.h \ + include/linux/nfs3.h \ + include/linux/nfs4.h \ + include/linux/nfs4_acl.h \ + include/linux/nfs4_mount.h \ + include/linux/nfs_fs.h \ + include/linux/nfs_fs_i.h \ + include/linux/nfs_fs_sb.h \ + include/linux/nfs_idmap.h \ + include/linux/nfs_mount.h \ + include/linux/nfs_page.h \ + include/linux/nfs_xdr.h \ + include/linux/nfsacl.h \ + include/linux/nfsd_idmap.h \ + include/linux/nfsd \ + include/linux/lockd \ + include/linux/sunrpc \ + fs/lockd \ + fs/nfs \ + fs/nfs_common \ + fs/nfsd \ + fs/nfsctl.c \ + net/sunrpc \ drivers/infiniband \ Documentation/infiniband \ ofed_scripts \ -- gitgui.0.8.4.g9c514 From YJia at tmriusa.com Thu Jul 10 15:53:40 2008 From: YJia at tmriusa.com (Yicheng Jia) Date: Thu, 10 Jul 2008 17:53:40 -0500 Subject: [ofa-general] RE: minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: <001301c8e2cb$36203ce0$e3d8180a@amr.corp.intel.com> Message-ID: > The opensm node requires ib_core, ib_mad, ib_umad, and ib_mthca. > The other nodes require ib_core, ib_mad, ib_mthca, plus others, > depending on you application. If it is a userspace application, then > you need ib_uverbs as well. Can the opensm resides in the end node which does the RDMA operation as well? The ib_core includes ib_sa and ib_cm modules, I use ib_core without these two modules, plus ib_mad and ib_mthca, in a single managed switch network, they work fine. Now I'm using a unmanaged switch with opensm, can I still use the previous driver modules (without ib_sa and ib_cm)? Of cause opensm need to be customized as well (no SA and CM?). The reason I'm doing this is I need to port opensm to QNX, I have to use as little modules as possible. Best, Yicheng "Sean Hefty" 07/10/2008 03:25 PM To "'Yicheng Jia'" , cc Subject RE: minimum sw components requirement for driver/opensm in a single unmanaged switch network > I have a IB network which consists of only a single unmanaged switch, > all end nodes connecting with the switch only need to do RDMA > read/write operation with each other. My question is, what are the > indispensable modules in driver's core and opensm that make the > network up and run? The opensm node requires ib_core, ib_mad, ib_umad, and ib_mthca. The other nodes require ib_core, ib_mad, ib_mthca, plus others, depending on you application. If it is a userspace application, then you need ib_uverbs as well. > I've been using only ib_mad module in driver's core with a managed > switch before, and the network works fine. So I assume that only the > ib_mad module in driver's core and SM in opensm are mandatory in my > network. The LIDs are assigned by them. The SA and CM modules are not > useful in my case. Am I right? You need to establish connections between the nodes, and configure the QPs. The QP number and LID information must be exchanged between the various nodes somehow (over a socket connection, manually entered into each application, user-defined MADs, whatever). If the machines also connect into an Ethernet network, you could exchange the data over sockets. - Sean _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jeffrey.C.Becker at nasa.gov Thu Jul 10 15:57:51 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Thu, 10 Jul 2008 15:57:51 -0700 Subject: [ofa-general] [Fwd: [PATCH] Add nfsrdma_mod to config/build options] Message-ID: <487693EF.8030005@nasa.gov> Hi all. I tried git-send-email to send this, and I thought it would give me a chance to edit the patch. This is the first patch to integrate NFSRDMA into OFED 1.4. I tested that it builds correctly with 2.6.26-rc6 (current OFED 1.4 base). Now onto the backports! -jeff Add nfsrdma_mod to config/build options Signed-off-by: Jeff Becker --- ofed_scripts/Makefile | 7 ++ ofed_scripts/configure | 170 +++++++++++++++++++++++++++++++++++++++++ ofed_scripts/makefile | 66 ++++++++++++++++ ofed_scripts/ofed_checkout.sh | 24 ++++++ 4 files changed, 267 insertions(+), 0 deletions(-) diff --git a/ofed_scripts/Makefile b/ofed_scripts/Makefile index dde593d..27b02eb 100644 --- a/ofed_scripts/Makefile +++ b/ofed_scripts/Makefile @@ -14,3 +14,10 @@ obj-$(CONFIG_CHELSIO_T3) += drivers/net/cxgb3/ obj-$(CONFIG_MLX4_CORE) += drivers/net/mlx4/ obj-$(CONFIG_RDS) += net/rds/ obj-$(CONFIG_MEMTRACK) += drivers/infiniband/debug/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += net/sunrpc/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += net/sunrpc/auth_gss +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += net/sunrpc/xprtrdma/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/nfs/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/lockd/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/nfs_common/ +obj-$(CONFIG_SUNRPC_XPRT_RDMA) += fs/nfsd/ diff --git a/ofed_scripts/configure b/ofed_scripts/configure index f468b85..2e31ee3 100755 --- a/ofed_scripts/configure +++ b/ofed_scripts/configure @@ -145,6 +145,9 @@ Usage: `basename $0` [options] --with-debug-info make CONFIG_DEBUG_INFO=y [yes] --without-debug-info [no] + --with-nfsrdma-mod make CONFIG_SUNRPC_XPRT_RDMA=m [no] + --without-nfsrdma-mod [yes] + --help - print out options @@ -506,6 +509,48 @@ main() --without-nes_debug-mod) CONFIG_INFINIBAND_NES_DEBUG= ;; + --with-nfsrdma-mod) + CONFIG_SUNRPC="m" + CONFIG_SUNRPC_GSS="m" + CONFIG_RPCSEC_GSS_KRB5="m" + CONFIG_RPCSEC_GSS_SPKM3="m" + CONFIG_SUNRPC_XPRT_RDMA="m" + CONFIG_NFS_FS="m" + CONFIG_NFS_V3="y" + CONFIG_NFS_V3_ACL="y" + CONFIG_NFS_ACL_SUPPORT="m" + CONFIG_NFS_DIRECTIO="y" + CONFIG_SYSCTL="y" + CONFIG_LOCKD="m" + CONFIG_LOCKD_V4="y" + CONFIG_NFSD="m" + CONFIG_NFSD_V2_ACL="y" + CONFIG_NFSD_V3="y" + CONFIG_NFSD_V3_ACL="y" + CONFIG_NFSD_V4="y" + CONFIG_NFSD_RDMA="y" + ;; + --without-nfsrdma-mod) + CONFIG_SUNRPC= + CONFIG_SUNRPC_GSS= + CONFIG_RPCSEC_GSS_KRB5= + CONFIG_RPCSEC_GSS_SPKM3= + CONFIG_SUNRPC_XPRT_RDMA= + CONFIG_NFS_FS= + CONFIG_NFS_V3= + CONFIG_NFS_V3_ACL= + CONFIG_NFS_ACL_SUPPORT= + CONFIG_NFS_DIRECTIO= + CONFIG_SYSCTL= + CONFIG_LOCKD= + CONFIG_LOCKD_V4= + CONFIG_NFSD= + CONFIG_NFSD_V2_ACL= + CONFIG_NFSD_V3= + CONFIG_NFSD_V3_ACL= + CONFIG_NFSD_V4= + CONFIG_NFSD_RDMA= + ;; --with-modprobe|--without-modprobe) ;; -h | --help) @@ -579,6 +624,29 @@ CONFIG_INFINIBAND_CXGB3=${CONFIG_INFINIBAND_CXGB3:-''} CONFIG_CHELSIO_T3=${CONFIG_CHELSIO_T3:-''} CONFIG_INFINIBAND_NES=${CONFIG_INFINIBAND_NES:-''} +CONFIG_SUNRPC_XPRT_RDMA=${CONFIG_SUNRPC_XPRT_RDMA:-''} +CONFIG_SUNRPC=${CONFIG_SUNRPC:-''} +CONFIG_SUNRPC_GSS=${CONFIG_SUNRPC_GSS:-''} +CONFIG_RPCSEC_GSS_KRB5=${CONFIG_RPCSEC_GSS_KRB5:-''} +CONFIG_RPCSEC_GSS_SPKM3=${CONFIG_RPCSEC_GSS_SPKM3:-''} + +CONFIG_NFS_FS=${CONFIG_NFS_FS:-''} +CONFIG_NFS_V3=${CONFIG_NFS_V3:-''} +CONFIG_NFS_V3_ACL=${CONFIG_NFS_V3_ACL:-''} +CONFIG_NFS_ACL_SUPPORT=${CONFIG_NFS_ACL_SUPPORT:-''} +CONFIG_NFS_DIRECTIO=${CONFIG_NFS_DIRECTIO:-''} +CONFIG_SYSCTL=${CONFIG_SYSCTL:-''} + +CONFIG_LOCKD=${CONFIG_LOCKD:-''} +CONFIG_LOCKD_V4=${CONFIG_LOCKD_V4:-''} + +CONFIG_NFSD=${CONFIG_NFSD:-''} +CONFIG_NFSD_V2_ACL=${CONFIG_NFSD_V2_ACL:-''} +CONFIG_NFSD_V3=${CONFIG_NFSD_V3:-''} +CONFIG_NFSD_V3_ACL=${CONFIG_NFSD_V3_ACL:-''} +CONFIG_NFSD_V4=${CONFIG_NFSD_V4:-''} +CONFIG_NFSD_RDMA=${CONFIG_NFSD_RDMA:-''} + CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=${CONFIG_INFINIBAND_IPOIB_DEBUG_DATA:-''} CONFIG_INFINIBAND_SDP_SEND_ZCOPY=${CONFIG_INFINIBAND_SDP_SEND_ZCOPY:-''} CONFIG_INFINIBAND_SDP_RECV_ZCOPY=${CONFIG_INFINIBAND_SDP_RECV_ZCOPY:-''} @@ -654,6 +722,27 @@ CONFIG_INFINIBAND_CXGB3=${CONFIG_INFINIBAND_CXGB3} CONFIG_CHELSIO_T3=${CONFIG_CHELSIO_T3} CONFIG_INFINIBAND_NES=${CONFIG_INFINIBAND_NES} +CONFIG_SUNRPC_XPRT_RDMA=${CONFIG_SUNRPC_XPRT_RDMA} +CONFIG_SUNRPC=${CONFIG_SUNRPC} +CONFIG_SUNRPC_GSS=${CONFIG_SUNRPC_GSS} +CONFIG_RPCSEC_GSS_KRB5=${CONFIG_RPCSEC_GSS_KRB5} +CONFIG_RPCSEC_GSS_SPKM3=${CONFIG_RPCSEC_GSS_SPKM3} + +CONFIG_NFS_FS=${CONFIG_NFS_FS} +CONFIG_NFS_V3=${CONFIG_NFS_V3} +CONFIG_NFS_V3_ACL=${CONFIG_NFS_V3_ACL} +CONFIG_NFS_ACL_SUPPORT=${CONFIG_NFS_ACL_SUPPORT} +CONFIG_NFS_DIRECTIO=${CONFIG_NFS_DIRECTIO} +CONFIG_SYSCTL=${CONFIG_SYSCTL} +CONFIG_LOCKD=${CONFIG_LOCKD} +CONFIG_LOCKD_V4=${CONFIG_LOCKD_V4} +CONFIG_NFSD=${CONFIG_NFSD} +CONFIG_NFSD_V2_ACL=${CONFIG_NFSD_V2_ACL} +CONFIG_NFSD_V3=${CONFIG_NFSD_V3} +CONFIG_NFSD_V3_ACL=${CONFIG_NFSD_V3_ACL} +CONFIG_NFSD_V4=${CONFIG_NFSD_V4} +CONFIG_NFSD_RDMA=${CONFIG_NFSD_RDMA} + CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=${CONFIG_INFINIBAND_IPOIB_DEBUG_DATA} CONFIG_INFINIBAND_SDP_SEND_ZCOPY=${CONFIG_INFINIBAND_SDP_SEND_ZCOPY} CONFIG_INFINIBAND_SDP_RECV_ZCOPY=${CONFIG_INFINIBAND_SDP_RECV_ZCOPY} @@ -875,6 +964,47 @@ if [ "X${CONFIG_INFINIBAND_AMSO1100}" == "Xm" ]; then else DEFINE_INFINIBAND_AMSO1100="#undef CONFIG_INFINIBAND_AMSO1100" fi +if [ "X${CONFIG_SUNRPC_XPRT_RDMA}" == "Xm" ]; then + DEFINE_CONFIG_SUNRPC_XPRT_RDMA="#define CONFIG_SUNRPC_XPRT_RDMA 1" + DEFINE_CONFIG_SUNRPC="#define CONFIG_SUNRPC 1" + DEFINE_CONFIG_SUNRPC_GSS="#define CONFIG_SUNRPC_GSS 1" + DEFINE_CONFIG_RPCSEC_GSS_KRB5="#define CONFIG_RPCSEC_GSS_KRB5 1" + DEFINE_CONFIG_RPCSEC_GSS_SPKM3="#define CONFIG_RPCSEC_GSS_SPKM3 1" + DEFINE_CONFIG_NFS_FS="#define CONFIG_NFS_FS 1" + DEFINE_CONFIG_NFS_V3="#define CONFIG_NFS_V3 1" + DEFINE_CONFIG_NFS_V3_ACL="#define CONFIG_NFS_V3_ACL 1" + DEFINE_CONFIG_NFS_ACL_SUPPORT="#define CONFIG_NFS_ACL_SUPPORT 1" + DEFINE_CONFIG_NFS_DIRECTIO="#define CONFIG_NFS_DIRECTIO 1" + DEFINE_CONFIG_SYSCTL="#define CONFIG_SYSCTL 1" + DEFINE_CONFIG_LOCKD="#define CONFIG_LOCKD 1" + DEFINE_CONFIG_LOCKD_V4="#define CONFIG_LOCKD_V4 1" + DEFINE_CONFIG_NFSD="#define CONFIG_NFSD 1" + DEFINE_CONFIG_NFSD_V2_ACL="#define CONFIG_NFSD_V2_ACL 1" + DEFINE_CONFIG_NFSD_V3="#define CONFIG_NFSD_V3 1" + DEFINE_CONFIG_NFSD_V3_ACL="#define CONFIG_NFSD_V3_ACL 1" + DEFINE_CONFIG_NFSD_V4="#define CONFIG_NFSD_V4 1" + DEFINE_CONFIG_NFSD_RDMA="#define CONFIG_NFSD_RDMA 1" +else + DEFINE_CONFIG_SUNRPC_XPRT_RDMA="#undef CONFIG_SUNRPC_XPRT_RDMA" + DEFINE_CONFIG_SUNRPC="#undef CONFIG_SUNRPC" + DEFINE_CONFIG_SUNRPC_GSS="#undef CONFIG_SUNRPC_GSS" + DEFINE_CONFIG_RPCSEC_GSS_KRB5="#undef CONFIG_RPCSEC_GSS_KRB5" + DEFINE_CONFIG_RPCSEC_GSS_SPKM3="#undef CONFIG_RPCSEC_GSS_SPKM3" + DEFINE_CONFIG_NFS_FS="#undef CONFIG_NFS_FS" + DEFINE_CONFIG_NFS_V3="#undef CONFIG_NFS_V3" + DEFINE_CONFIG_NFS_V3_ACL="#undef CONFIG_NFS_V3_ACL" + DEFINE_CONFIG_NFS_ACL_SUPPORT="#undef CONFIG_NFS_ACL_SUPPORT" + DEFINE_CONFIG_NFS_DIRECTIO="#undef CONFIG_NFS_DIRECTIO" + DEFINE_CONFIG_SYSCTL="#undef CONFIG_SYSCTL" + DEFINE_CONFIG_LOCKD="#undef CONFIG_LOCKD" + DEFINE_CONFIG_LOCKD_V4="#undef CONFIG_LOCKD_V4" + DEFINE_CONFIG_NFSD="#undef CONFIG_NFSD" + DEFINE_CONFIG_NFSD_V2_ACL="#undef CONFIG_NFSD_V2_ACL" + DEFINE_CONFIG_NFSD_V3="#undef CONFIG_NFSD_V3" + DEFINE_CONFIG_NFSD_V3_ACL="#undef CONFIG_NFSD_V3_ACL" + DEFINE_CONFIG_NFSD_V4="#undef CONFIG_NFSD_V4" + DEFINE_CONFIG_NFSD_RDMA="#undef CONFIG_NFSD_RDMA" +fi cat >> ${AUTOCONF_H} << EOFAUTOCONF #undef CONFIG_MEMTRACK #undef CONFIG_DEBUG_INFO @@ -913,6 +1043,26 @@ cat >> ${AUTOCONF_H} << EOFAUTOCONF #undef CONFIG_INFINIBAND_NES #undef CONFIG_INFINIBAND_NES_DEBUG +#undef CONFIG_SUNRPC_XPRT_RDMA +#undef CONFIG_SUNRPC +#undef CONFIG_SUNRPC_GSS +#undef CONFIG_RPCSEC_GSS_KRB5 +#undef CONFIG_RPCSEC_GSS_SPKM3 +#undef CONFIG_NFS_FS +#undef CONFIG_NFS_V3 +#undef CONFIG_NFS_V3_ACL +#undef CONFIG_NFS_ACL_SUPPORT +#undef CONFIG_NFS_DIRECTIO +#undef CONFIG_SYSCTL +#undef CONFIG_LOCKD +#undef CONFIG_LOCKD_V4 +#undef CONFIG_NFSD +#undef CONFIG_NFSD_V2_ACL +#undef CONFIG_NFSD_V3 +#undef CONFIG_NFSD_V3_ACL +#undef CONFIG_NFSD_V4 +#undef CONFIG_NFSD_RDMA + #undef CONFIG_INFINIBAND_IPOIB_DEBUG_DATA #undef CONFIG_INFINIBAND_SDP_SEND_ZCOPY #undef CONFIG_INFINIBAND_SDP_RECV_ZCOPY @@ -939,6 +1089,26 @@ ${DEFINE_INFINIBAND_CXGB3} ${DEFINE_CHELSIO_T3} ${DEFINE_INFINIBAND_NES} +${DEFINE_CONFIG_SUNRPC_XPRT_RDMA} +${DEFINE_CONFIG_SUNRPC} +${DEFINE_CONFIG_SUNRPC_GSS} +${DEFINE_CONFIG_RPCSEC_GSS_KRB5} +${DEFINE_CONFIG_RPCSEC_GSS_SPKM3} +${DEFINE_CONFIG_NFS_FS} +${DEFINE_CONFIG_NFS_V3} +${DEFINE_CONFIG_NFS_V3_ACL} +${DEFINE_CONFIG_NFS_ACL_SUPPORT} +${DEFINE_CONFIG_NFS_DIRECTIO} +${DEFINE_CONFIG_SYSCTL} +${DEFINE_CONFIG_LOCKD} +${DEFINE_CONFIG_LOCKD_V4} +${DEFINE_CONFIG_NFSD} +${DEFINE_CONFIG_NFSD_V2_ACL} +${DEFINE_CONFIG_NFSD_V3} +${DEFINE_CONFIG_NFSD_V3_ACL} +${DEFINE_CONFIG_NFSD_V4} +${DEFINE_CONFIG_NFSD_RDMA} + ${DEFINE_INFINIBAND_IPOIB_DEBUG} ${DEFINE_INFINIBAND_ISER} ${DEFINE_SCSI_ISCSI_ATTRS} diff --git a/ofed_scripts/makefile b/ofed_scripts/makefile index c4d2143..5f8cdf6 100644 --- a/ofed_scripts/makefile +++ b/ofed_scripts/makefile @@ -5,6 +5,7 @@ all: .PHONY: install_kernel_iscsi uninstall_iscsi .PHONY: install_kernel_cxgb3 uninstall_cxgb3 .PHONY: install_kernel_rds uninstall_rds +.PHONY: install_kernel_xprtrdma uninstall_xprtrdma .PHONY: install_kernel_mlx4 uninstall_mlx4 .DELETE_ON_ERROR: @@ -41,6 +42,11 @@ install_kernel: install_kernel_cxgb3 uninstall: uninstall_cxgb3 endif +ifeq ($(CONFIG_SUNRPC_XPRT_RDMA),m) +install_kernel: install_kernel_xprtrdma +uninstall: uninstall_xprtrdma +endif + ifeq ($(CONFIG_RDS),m) install_kernel: install_kernel_rds uninstall: uninstall_rds @@ -116,6 +122,25 @@ kernel: CONFIG_MLX4_ETHERNET=$(CONFIG_MLX4_ETHERNET) \ CONFIG_MLX4_DEBUG=$(CONFIG_MLX4_DEBUG) \ CONFIG_INFINIBAND_AMSO1100=$(CONFIG_INFINIBAND_AMSO1100) \ + CONFIG_SUNRPC_XPRT_RDMA=$(CONFIG_SUNRPC_XPRT_RDMA) \ + CONFIG_SUNRPC=$(CONFIG_SUNRPC) \ + CONFIG_SUNRPC_GSS=$(CONFIG_SUNRPC_GSS) \ + CONFIG_RPCSEC_GSS_KRB5=$(CONFIG_RPCSEC_GSS_KRB5) \ + CONFIG_RPCSEC_GSS_SPKM3=$(CONFIG_RPCSEC_GSS_SPKM3) \ + CONFIG_NFS_FS=$(CONFIG_NFS_FS) \ + CONFIG_NFS_V3=$(CONFIG_NFS_V3) \ + CONFIG_NFS_V3_ACL=$(CONFIG_NFS_V3_ACL) \ + CONFIG_NFS_ACL_SUPPORT=$(CONFIG_NFS_ACL_SUPPORT) \ + CONFIG_NFS_DIRECTIO=$(CONFIG_NFS_DIRECTIO) \ + CONFIG_SYSCTL=$(CONFIG_SYSCTL) \ + CONFIG_LOCKD=$(CONFIG_LOCKD) \ + CONFIG_LOCKD_V4=$(CONFIG_LOCKD_V4) \ + CONFIG_NFSD=$(CONFIG_NFSD) \ + CONFIG_NFSD_V2_ACL=$(CONFIG_NFSD_V2_ACL) \ + CONFIG_NFSD_V3=$(CONFIG_NFSD_V3) \ + CONFIG_NFSD_V3_ACL=$(CONFIG_NFSD_V3_ACL) \ + CONFIG_NFSD_V4=$(CONFIG_NFSD_V4) \ + CONFIG_NFSD_RDMA=$(CONFIG_NFSD_RDMA) \ LINUXINCLUDE=' \ -include include/linux/autoconf.h \ -include $(CWD)/include/linux/autoconf.h \ @@ -212,6 +237,39 @@ install_kernel_iscsi: install_modules if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION);fi; +install_kernel_nfsrdma: install_modules + # backup all the modules + NFSRDMA_PATHS="fs/nfs fs/nfsd fs/lockd fs/nfs_common net/sunrpc net/sunrpc/xprtrdma net/sunrpc/auth_gss"; \ + if [ -d $(DESTDIR)/$(MODULES_DIR)/kernel/drivers/net/sunrpc ]; then \ + mkdir -p $(DESTDIR)/$(prefix)/backup; \ + if [ ! -f $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz ]; then \ + NFSRDMA_FULL_PATH=""; \ + for this_path in $$NFSRDMA_PATHS; do \ + NFSRDMA_FULL_PATH="$(DESTDIR)/$(MODULES_DIR)/kernel/$$this_path $$NFSRDMA_FULL_PATH"; \ + done; \ + tar czfP $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz $$NFSRDMA_FULL_PATH; \ + fi; \ + fi; + # Copy new nfs rdma kernel modules to $(DESTDIR)/$(MODULES_DIR)/kernel + NFSRDMA_MODS="net/sunrpc/auth_gss/auth_rpcgss.ko net/sunrpc/auth_gss/rpcsec_gss_krb5.ko net/sunrpc/auth_gss/rpcsec_gss_spkm3.ko fs/lockd/lockd.ko fs/nfs_common/nfs_acl.ko fs/nfsd/nfsd.ko fs/nfs/nfs.ko net/sunrpc/sunrpc.ko net/sunrpc/xprtrdma/svcrdma.ko net/sunrpc/xprtrdma/xprtrdma.ko"; \ + if [ -d $(CWD)/lib/modules/$(KVERSION)/net/sunrpc/xprtrdma ]; then \ + for each_path in $$NFSRDMA_PATHS; do \ + mkdir -p $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_path; \ + done; \ + for each_module in $$NFSRDMA_MODS; do \ + mv $(CWD)/lib/modules/$(KVERSION)/$$each_module $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_module; \ + done; \ + fi; \ + if [ -d $(CWD)/lib/modules/$(KVERSION)/extra/net/sunrpc ]; then \ + for each_path in $$NFSRDMA_PATHS; do \ + mkdir -p $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_path; \ + done; \ + for each_module in $$NFSRDMA_MODS; do \ + mv $(CWD)/lib/modules/$(KVERSION)/extra/$$each_module $(DESTDIR)/$(MODULES_DIR)/kernel/$$each_module; \ + done; \ + fi; + if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION); fi; + install_kernel_cxgb3: install_modules # Backup original cxgb3.ko kernel module under 'prefix'/backup directory if [ -d $(DESTDIR)/$(MODULES_DIR)/kernel/drivers/net/cxgb3 ]; then \ @@ -311,6 +369,14 @@ uninstall_iscsi: if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION);fi; +uninstall_nfsrdma: + if [ -f $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz ]; then \ + tar xzfP $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz; \ + /bin/rm -f $(DESTDIR)/$(prefix)/backup/nfsrdma_ko-$(KVERSION).tgz; \ + fi; + + if [ ! -n "$(DESTDIR)" ]; then $(DEPMOD) -r -ae $(KVERSION);fi; + uninstall_cxgb3: if [ -f $(DESTDIR)/$(prefix)/backup/cxgb3_ko-$(KVERSION).tgz ]; then \ /bin/rm $(DESTDIR)/$(MODULES_DIR)/kernel/drivers/net/cxgb3/cxgb3.ko ; \ diff --git a/ofed_scripts/ofed_checkout.sh b/ofed_scripts/ofed_checkout.sh index f3286a4..f98a09c 100755 --- a/ofed_scripts/ofed_checkout.sh +++ b/ofed_scripts/ofed_checkout.sh @@ -17,6 +17,30 @@ ex git checkout $1 `git-ls-tree -r --name-only $1 \ include/rdma \ include/scsi/srp.h \ include/scsi/iscsi_proto.h \ + include/linux/nfs.h \ + include/linux/nfs2.h \ + include/linux/nfs3.h \ + include/linux/nfs4.h \ + include/linux/nfs4_acl.h \ + include/linux/nfs4_mount.h \ + include/linux/nfs_fs.h \ + include/linux/nfs_fs_i.h \ + include/linux/nfs_fs_sb.h \ + include/linux/nfs_idmap.h \ + include/linux/nfs_mount.h \ + include/linux/nfs_page.h \ + include/linux/nfs_xdr.h \ + include/linux/nfsacl.h \ + include/linux/nfsd_idmap.h \ + include/linux/nfsd \ + include/linux/lockd \ + include/linux/sunrpc \ + fs/lockd \ + fs/nfs \ + fs/nfs_common \ + fs/nfsd \ + fs/nfsctl.c \ + net/sunrpc \ drivers/infiniband \ Documentation/infiniband \ ofed_scripts \ -- gitgui.0.8.4.g9c514 From sean.hefty at intel.com Thu Jul 10 16:07:06 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 10 Jul 2008 16:07:06 -0700 Subject: [ofa-general] RE: minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: References: <001301c8e2cb$36203ce0$e3d8180a@amr.corp.intel.com> Message-ID: <000001c8e2e1$ac3df640$e3d8180a@amr.corp.intel.com> > Can the opensm resides in the end node which does the RDMA operation > as well? yes > The ib_core includes ib_sa and ib_cm modules, I use ib_core without > these two modules, plus ib_mad and ib_mthca, in a single managed > switch network, they work fine. Now I'm using a unmanaged switch with > opensm, can I still use the previous driver modules (without ib_sa and > ib_cm)? Of cause opensm need to be customized as well (no SA and CM?). I was referring to actual modules, with ib_core being separate from ib_sa and ib_cm. You can make due without ib_sa and ib_cm, provided that you get similar functionality from another source. (I.e. get path information and exchange QP data.) - Sean From weiny2 at llnl.gov Thu Jul 10 16:09:51 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 10 Jul 2008 16:09:51 -0700 Subject: [ofa-general] Re: [PATCH RFC] opensm/event_plugin: plugin API version 2 In-Reply-To: <1215636386.18371.126.camel@hrosenstock-ws.xsigo.com> References: <1214501218.13056.620.camel@hrosenstock-ws.xsigo.com> <20080627181931.GO29456@sashak.voltaire.com> <20080627130333.08d783c6.weiny2@llnl.gov> <20080627203745.GA29198@obsidianresearch.com> <20080627230339.GV29456@sashak.voltaire.com> <1214825806.3486.173.camel@hrosenstock-ws.xsigo.com> <20080630142705.GF7229@sashak.voltaire.com> <1214861294.3486.216.camel@hrosenstock-ws.xsigo.com> <20080702131104.GI30305@sashak.voltaire.com> <1215005835.3486.269.camel@hrosenstock-ws.xsigo.com> <20080709040320.GC12551@sashak.voltaire.com> <1215636386.18371.126.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080710160951.4faef4e6.weiny2@llnl.gov> On Wed, 09 Jul 2008 13:46:26 -0700 Hal Rosenstock wrote: > On Wed, 2008-07-09 at 07:03 +0300, Sasha Khapyorsky wrote: > > On 06:37 Wed 02 Jul , Hal Rosenstock wrote: > > > > > > This is related to what I wrote below about the OpenFabrics licensing > > > > > requirements. The idea is if GPL licensing were to be allowed (perhaps > > > > > only in some limited context), then there could be two different > > > > > packages: dual and GPL. In that way plugins would be more assured of > > > > > being compatible with each other and OpenSM. > > > > > > > > I would prefer to separate packages by its functionality and not due to > > > > licensing issues. > > > > > > Right, licensed based packages do put all related functionality in a > > > similar bucket (management) but is that the basis to make such a > > > decision ? > > > > Which decisions? I'm not following. > > The decision to expose all internals in this manner as well as the > decision not to see what license modifications might be possible. > Just to be clear I don't think having a generalized plugin is being motivated by licensing issues (At least not for me it isn't). I think the issue is more a matter of distribution of functionality and ability to extend OpenSM without affecting other users or more importantly the OFED build and install. I had this long email written but let me try and be brief. I think what I like with Sashas proposal is how similar it is to the kernel. Plugins can be added and loaded separate from OpenSM. If some functionality becomes "standard" and Sasha wants to integrate it to the management git tree, it could stay as a plugin (like kernel module) but be included with the OpenSM packages. I don't think that is a bad model. Some plugins might become obsolete by interface changes but that would actually encourage people to add them to the official build (read "git tree")_if_ _possible_. Furthermore, now that I think about it, this could solve the problem of using MySQL with the current plugin. If the plugin is integrated and compiled by default, but not installed by the user they could run _without_ MySQL installed!!! However, it would be "automatically available" (through OFED rpm install) to any user without them having to recompile anything!!! I like that! Ira From taylor at hpc.ufl.edu Thu Jul 10 16:15:21 2008 From: taylor at hpc.ufl.edu (Charles Taylor) Date: Thu, 10 Jul 2008 19:15:21 -0400 Subject: [ofa-general] Cisco 7008 Internal/External Port Mappings In-Reply-To: <1215725788.18371.193.camel@hrosenstock-ws.xsigo.com> References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> <5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu> <1215725788.18371.193.camel@hrosenstock-ws.xsigo.com> Message-ID: Nope. Looks pretty much the same as ibnetdiscover without the "- g". It would be really useful if it *did* work. :) Thanks for the suggestion though. I think we just may be stuck mapping it empirically. I'm sure the pattern will become obvious quickly enough. Charlie Taylor UF HPC Center On Jul 10, 2008, at 5:36 PM, Hal Rosenstock wrote: > On Thu, 2008-07-10 at 17:27 -0400, Charles Taylor wrote: >> We've got a couple of Cisco 7008/Topspin 270 core switches. Much >> of >> the diagnostic and topology information we get from the ib diag tools >> in OFED refer to the internal ports rather than the actual external >> slot/port. > > Does ibnetdiscover -g work ? There was some work done to do this > mapping > for Cisco but it may be out of date. > > -- Hal > >> We could deduce the mappings, I suppose, by tracing >> cables and using ibtracert between nodes but that seems like a real >> pain. I was wondering if anyone on this list knows how that >> mapping >> works. The chassis itself consists of 8 "ib port" blades in the >> front labelled as slots 1 - 8. There are then six "fabric >> controller" slots in the back of the chassis labelled 9 - 14. >> Slots 11 and 12 are "special" as they hold the "master" and "standby >> master" fabric controllers. >> >> Seems like the FC's in slots 11 and 12 connect the the FCs in slots >> 9,10,13,14 which in turn connect to the external port blades in slots >> 1 - 8. >> >> Again, all of the diagnostic information refers to the internal FC >> ports and mapping those to the external ports (to which we actually >> connect leaf switches) seems non-trivial. I think there was even a >> document somewhere that described the mapping. We just can't find >> it >> now. >> >> Thanks, >> >> Charlie Taylor >> UF HPC Center >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From YJia at tmriusa.com Thu Jul 10 16:39:48 2008 From: YJia at tmriusa.com (Yicheng Jia) Date: Thu, 10 Jul 2008 18:39:48 -0500 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: Message-ID: > If you want to avoid all the SM stuff, and are willing to program the switches directly (a few mads) Is it done by opensm? What information should be set up in the switch by opensm? > Then to figure out QP connections, you just use a function of 3 parameters: > my_qp_num = fn_sqp(my_node, target_node, qp_num) > target_qp_num = fn_tqp(my_node, target_node, qp_num) > Where qp_num is a small number between 0 and the maximum number of QPs you need active between any 2 endpoints. Can the qp_num be manually assigned? Does it need opensm be involved? > If it works, you are done. If not, reset, up, wait for him to connect and send something to you. Is it reliable? I mean the QPs connection will keep alive during the QPs lifecycle? Best, Yicheng "Jim Mott" 07/10/2008 04:17 PM To "Yicheng Jia" , cc Subject RE: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network If you want to avoid all the SM stuff, and are willing to program the switches directly (a few mads), then I’ve used schemes like: Node LID=base + (switch port * constant) (base=0, constant = 1 works) Then to figure out QP connections, you just use a function of 3 parameters: my_qp_num = fn_sqp(my_node, target_node, qp_num) target_qp_num = fn_tqp(my_node, target_node, qp_num) Where qp_num is a small number between 0 and the maximum number of QPs you need active between any 2 endpoints. With the above scheme, you know your node_id (switch port number), your lid, the lid of the target node, and the QPs on both sides. From there on, it is clear sailing. You don’t even need to send MADs; just transition the QP up and try and use it. If it works, you are done. If not, reset, up, wait for him to connect and send something to you. A little timer to make sure everybody retries once in awhile and what can go wrong? Jim From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia Sent: Thursday, July 10, 2008 2:59 PM To: general at lists.openfabrics.org Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network Hi Folks, I have a IB network which consists of only a single unmanaged switch, all end nodes connecting with the switch only need to do RDMA read/write operation with each other. My question is, what are the indispensable modules in driver's core and opensm that make the network up and run? I've been using only ib_mad module in driver's core with a managed switch before, and the network works fine. So I assume that only the ib_mad module in driver's core and SM in opensm are mandatory in my network. The LIDs are assigned by them. The SA and CM modules are not useful in my case. Am I right? I need to minimize driver and opensm to fit them in my network, the HCA driver is mthca. Best, Yicheng _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From YJia at tmriusa.com Thu Jul 10 16:47:42 2008 From: YJia at tmriusa.com (Yicheng Jia) Date: Thu, 10 Jul 2008 18:47:42 -0500 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: <1215725123.18371.185.camel@hrosenstock-ws.xsigo.com> Message-ID: > You would also need to walk the ports to active (via MADs) after setting > the LIDs as well as setup the switch forwarding table (LFT). Are these done by opensm? Are these only two tasks that need to be done by opensm in a single unmanaged switch network? Best, Yicheng Hal Rosenstock 07/10/2008 04:39 PM To Jim Mott cc Yicheng Jia , general at lists.openfabrics.org Subject RE: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network On Thu, 2008-07-10 at 14:17 -0700, Jim Mott wrote: > If you want to avoid all the SM stuff, and are willing to program the > switches directly (a few mads), You would also need to walk the ports to active (via MADs) after setting the LIDs as well as setup the switch forwarding table (LFT). -- Hal > then I’ve used schemes like: > > > > Node LID=base + (switch port * constant) (base=0, constant = 1 works) > > > > Then to figure out QP connections, you just use a function of 3 > parameters: > > my_qp_num = fn_sqp(my_node, target_node, qp_num) > > target_qp_num = fn_tqp(my_node, target_node, qp_num) > > Where qp_num is a small number between 0 and the maximum number of QPs > you need active between any 2 endpoints. > > > > With the above scheme, you know your node_id (switch port number), > your lid, the lid of the target node, and the QPs on both sides. > From there on, it is clear sailing. You don’t even need to send MADs; > just transition the QP up and try and use it. If it works, you are > done. If not, reset, up, wait for him to connect and send something > to you. A little timer to make sure everybody retries once in awhile > and what can go wrong? > > > > Jim > > From:general-bounces at lists.openfabrics.org [mailto:general- > bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia > Sent: Thursday, July 10, 2008 2:59 PM > To: general at lists.openfabrics.org > Subject: [ofa-general] minimum sw components requirement for > driver/opensm in a single unmanaged switch network > > > > > > Hi Folks, > > I have a IB network which consists of only a single unmanaged switch, > all end nodes connecting with the switch only need to do RDMA > read/write operation with each other. My question is, what are the > indispensable modules in driver's core and opensm that make the > network up and run? > > I've been using only ib_mad module in driver's core with a managed > switch before, and the network works fine. So I assume that only the > ib_mad module in driver's core and SM in opensm are mandatory in my > network. The LIDs are assigned by them. The SA and CM modules are not > useful in my case. Am I right? > > I need to minimize driver and opensm to fit them in my network, the > HCA driver is mthca. > > Best, > Yicheng > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by > MessageLabs. For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ _____________________________________________________________________________ Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com _____________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From weiny2 at llnl.gov Thu Jul 10 17:44:07 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 10 Jul 2008 17:44:07 -0700 Subject: [ofa-general] [PATCH v2] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop Message-ID: <20080710174407.5ab1e4f2.weiny2@llnl.gov> >From fa67a47678e9127f5253b909b1a561fdccaf6609 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 1 Jul 2008 14:31:00 -0700 Subject: [PATCH] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop Signed-off-by: Ira K. Weiny --- infiniband-diags/src/saquery.c | 117 ++++++++++++++++------------------------ 1 files changed, 46 insertions(+), 71 deletions(-) diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 1594cad..6e7d16a 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -193,13 +193,13 @@ print_node_record(ib_node_record_t *node_record) static void dump_path_record(void *data) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; ib_path_rec_t *p_pr = data; printf("PathRecord dump:\n" "\t\tservice_id..............0x%016" PRIx64 "\n" - "\t\tdgid....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\tsgid....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tdgid....................%s\n" + "\t\tsgid....................%s\n" "\t\tdlid....................0x%X\n" "\t\tslid....................0x%X\n" "\t\thop_flow_raw............0x%X\n" @@ -216,10 +216,8 @@ static void dump_path_record(void *data) "\t\tresv3...................0x%X\n" "", cl_ntoh64( p_pr->service_id ), - cl_ntoh64( p_pr->dgid.unicast.prefix ), - cl_ntoh64( p_pr->dgid.unicast.interface_id ), - cl_ntoh64( p_pr->sgid.unicast.prefix ), - cl_ntoh64( p_pr->sgid.unicast.interface_id ), + inet_ntop(AF_INET6, p_pr->dgid.raw, gid_str, sizeof gid_str), + inet_ntop(AF_INET6, p_pr->sgid.raw, gid_str2, sizeof gid_str2), cl_ntoh16( p_pr->dlid ), cl_ntoh16( p_pr->slid ), cl_ntoh32( p_pr->hop_flow_raw ), @@ -237,32 +235,10 @@ static void dump_path_record(void *data) ); } -/** - * str must be longer than 32 to hold the full gid. - * len will be checked to ensure this. - */ -static char * -sprint_gid(ib_gid_t *gid, char *str, size_t len) -{ - int i = 0; - char tmp[16]; - - assert(str != NULL); - assert(len > 32); - - str[0] = '\0'; - for (i = 0; i < 16; i++) { - sprintf(tmp, "%02X", gid->raw[i]); - strcat(str, tmp); - } - - return (str); -} - static void dump_class_port_info(void *data) { - size_t GID_STR_LEN = 256; - char gid_str[GID_STR_LEN]; + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; ib_class_port_info_t *class_port_info = data; printf("SA ClassPortInfo:\n" @@ -271,13 +247,13 @@ static void dump_class_port_info(void *data) "\t\tCapability mask..........0x%04X\n" "\t\tCapability mask 2........0x%08X\n" "\t\tResponse time value......0x%02X\n" - "\t\tRedirect GID.............0x%s\n" + "\t\tRedirect GID.............%s\n" "\t\tRedirect TC/SL/FL........0x%08X\n" "\t\tRedirect LID.............0x%04X\n" "\t\tRedirect PKey............0x%04X\n" "\t\tRedirect QP..............0x%08X\n" "\t\tRedirect QKey............0x%08X\n" - "\t\tTrap GID.................0x%s\n" + "\t\tTrap GID.................%s\n" "\t\tTrap TC/SL/FL............0x%08X\n" "\t\tTrap LID.................0x%04X\n" "\t\tTrap PKey................0x%04X\n" @@ -289,13 +265,15 @@ static void dump_class_port_info(void *data) cl_ntoh16(class_port_info->cap_mask), ib_class_cap_mask2(class_port_info), ib_class_resp_time_val(class_port_info), - sprint_gid(&(class_port_info->redir_gid), gid_str, GID_STR_LEN), + inet_ntop(AF_INET6, &(class_port_info->redir_gid), gid_str, + sizeof gid_str), cl_ntoh32(class_port_info->redir_tc_sl_fl), cl_ntoh16(class_port_info->redir_lid), cl_ntoh16(class_port_info->redir_pkey), cl_ntoh32(class_port_info->redir_qp), cl_ntoh32(class_port_info->redir_qkey), - sprint_gid(&(class_port_info->trap_gid), gid_str, GID_STR_LEN), + inet_ntop(AF_INET6, &(class_port_info->trap_gid), gid_str2, + sizeof gid_str2), cl_ntoh32(class_port_info->trap_tc_sl_fl), cl_ntoh16(class_port_info->trap_lid), cl_ntoh16(class_port_info->trap_pkey), @@ -326,20 +304,19 @@ static void dump_portinfo_record(void *data) static void dump_multicast_group_record(void *data) { + char gid_str[INET6_ADDRSTRLEN]; ib_member_rec_t *p_mcmr = data; uint8_t sl; ib_member_get_sl_flow_hop(p_mcmr->sl_flow_hop, &sl, NULL, NULL); printf("MCMemberRecord group dump:\n" - "\t\tMGID....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tMGID....................%s\n" "\t\tMlid....................0x%X\n" "\t\tMtu.....................0x%X\n" "\t\tpkey....................0x%X\n" "\t\tRate....................0x%X\n" "\t\tSL......................0x%X\n" "", - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, sizeof gid_str), cl_ntoh16( p_mcmr->mlid ), p_mcmr->mtu, cl_ntoh16( p_mcmr->pkey ), @@ -350,8 +327,9 @@ static void dump_multicast_group_record(void *data) static void dump_multicast_member_record(void *data) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; ib_member_rec_t *p_mcmr = data; - uint64_t gid_prefix = cl_ntoh64( p_mcmr->port_gid.unicast.prefix ); uint64_t gid_interface_id = cl_ntoh64( p_mcmr->port_gid.unicast.interface_id ); uint16_t mlid = cl_ntoh16( p_mcmr->mlid ); int i = 0; @@ -371,29 +349,26 @@ static void dump_multicast_member_record(void *data) if (requested_name) { if (strtol(requested_name, NULL, 0) == mlid) { - printf("\t\tPortGid.................0x%016" PRIx64 " : " - "0x%016" PRIx64 " (%s)\n", - gid_prefix, - gid_interface_id, + printf("\t\tPortGid.................%s (%s)\n", + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, + gid_str, sizeof gid_str), node_name ); } } else { printf("MCMemberRecord member dump:\n" - "\t\tMGID....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tMGID....................%s\n" "\t\tMlid....................0x%X\n" - "\t\tPortGid.................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tPortGid.................%s\n" "\t\tScopeState..............0x%X\n" "\t\tProxyJoin...............0x%X\n" "\t\tNodeDescription.........%s\n" "", - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16( p_mcmr->mlid ), - gid_prefix, - gid_interface_id, + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, + gid_str2, sizeof gid_str2), p_mcmr->scope_state, p_mcmr->proxy_join, node_name @@ -403,6 +378,7 @@ static void dump_multicast_member_record(void *data) static void dump_service_record(void *data) { + char gid_str[INET6_ADDRSTRLEN]; char buf_service_key[35]; char buf_service_name[65]; ib_service_record_t *p_sr = data; @@ -430,8 +406,7 @@ static void dump_service_record(void *data) printf("ServiceRecord dump:\n" "\t\tServiceID...............0x%016" PRIx64 "\n" - "\t\tServiceGID..............0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tServiceGID..............%s\n" "\t\tServiceP_Key............0x%X\n" "\t\tServiceLease............0x%X\n" "\t\tServiceKey..............%s\n" @@ -468,8 +443,8 @@ static void dump_service_record(void *data) "\t\tServiceData64.2.........0x%016" PRIx64 "\n" "", cl_ntoh64( p_sr->service_id ), - cl_ntoh64( p_sr->service_gid.unicast.prefix ), - cl_ntoh64( p_sr->service_gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_sr->service_gid.raw, gid_str, + sizeof gid_str), cl_ntoh16( p_sr->service_pkey ), cl_ntoh32( p_sr->service_lease ), buf_service_key, @@ -501,6 +476,8 @@ static void dump_service_record(void *data) static void dump_inform_info_record(void *data) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; ib_inform_info_record_t *p_iir = data; uint32_t qpn; uint8_t resp_time_val; @@ -510,11 +487,10 @@ static void dump_inform_info_record(void *data) if (p_iir->inform_info.is_generic) { printf("InformInfoRecord dump:\n" "\t\tRID\n" - "\t\tSubscriberGID...........0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tSubscriberGID...........%s\n" "\t\tSubscriberEnum..........0x%X\n" "\t\tInformInfo dump:\n" - "\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" + "\t\tgid.....................%s\n" "\t\tlid_range_begin.........0x%X\n" "\t\tlid_range_end...........0x%X\n" "\t\tis_generic..............0x%X\n" @@ -525,11 +501,11 @@ static void dump_inform_info_record(void *data) "\t\tresp_time_val...........0x%X\n" "\t\tnode_type...............0x%06X\n" "", - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, gid_str, + sizeof gid_str), cl_ntoh16( p_iir->subscriber_enum ), - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, gid_str2, + sizeof gid_str2), cl_ntoh16( p_iir->inform_info.lid_range_begin ), cl_ntoh16( p_iir->inform_info.lid_range_end ), p_iir->inform_info.is_generic, @@ -543,11 +519,10 @@ static void dump_inform_info_record(void *data) } else { printf("InformInfoRecord dump:\n" "\t\tRID\n" - "\t\tSubscriberGID...........0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\tSubscriberGID...........%s\n" "\t\tSubscriberEnum..........0x%X\n" "\t\tInformInfo dump:\n" - "\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" + "\t\tgid.....................%s\n" "\t\tlid_range_begin.........0x%X\n" "\t\tlid_range_end...........0x%X\n" "\t\tis_generic..............0x%X\n" @@ -558,11 +533,11 @@ static void dump_inform_info_record(void *data) "\t\tresp_time_val...........0x%X\n" "\t\tvendor_id...............0x%06X\n" "", - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, gid_str, + sizeof gid_str), cl_ntoh16( p_iir->subscriber_enum ), - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, + gid_str2, sizeof gid_str2), cl_ntoh16( p_iir->inform_info.lid_range_begin ), cl_ntoh16( p_iir->inform_info.lid_range_end ), p_iir->inform_info.is_generic, -- 1.5.4.5 From weiny2 at llnl.gov Thu Jul 10 17:44:08 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 10 Jul 2008 17:44:08 -0700 Subject: [ofa-general] [PATCH v2] infiniband-diags/src/ibaddr.c: convert GID prints to use inet_ntop Message-ID: <20080710174408.10a73317.weiny2@llnl.gov> >From a799f458b9368e99894a38ebf7f88bbbf4057fc9 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 1 Jul 2008 15:48:54 -0700 Subject: [PATCH] infiniband-diags/src/ibaddr.c: convert GID prints to use inet_ntop Signed-off-by: Ira K. Weiny --- infiniband-diags/src/ibaddr.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c index 6c56a3e..591654f 100644 --- a/infiniband-diags/src/ibaddr.c +++ b/infiniband-diags/src/ibaddr.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include @@ -52,6 +53,7 @@ char *argv0 = "ibaddr"; static int ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) { + char gid_str[INET6_ADDRSTRLEN]; uint8_t portinfo[64]; uint8_t nodeinfo[64]; uint64_t guid, prefix; @@ -74,7 +76,8 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) mad_encode_field(gid, IB_GID_GUID_F, &guid); if (show_gid) { - printf("GID 0x%s%s ", mad_dump_val(IB_GID_PREFIX_F, buf1, sizeof buf1, &prefix), mad_dump_val(IB_GID_GUID_F, buf2, sizeof buf2, &guid)); + printf("GID %s ", inet_ntop(AF_INET6, gid, gid_str, + sizeof gid_str)); } if (show_lid > 0) -- 1.5.4.5 From weiny2 at llnl.gov Thu Jul 10 17:44:10 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 10 Jul 2008 17:44:10 -0700 Subject: [ofa-general] [PATCH] OpenSM: convert GID prints to use inet_ntop Message-ID: <20080710174410.1261bc4f.weiny2@llnl.gov> >From 5638ddc23276e0d83179d01fb0ee7bcd735f4ab0 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 10 Jul 2008 17:35:37 -0700 Subject: [PATCH] OpenSM: convert GID prints to use inet_ntop Signed-off-by: Ira K. Weiny --- opensm/opensm/osm_helper.c | 62 ++++---- opensm/opensm/osm_inform.c | 27 ++-- opensm/opensm/osm_perfmgr.c | 8 +- opensm/opensm/osm_sa_informinfo.c | 8 +- opensm/opensm/osm_sa_mcmember_record.c | 104 ++++++------- opensm/opensm/osm_sa_path_record.c | 8 +- opensm/osmtest/osmt_multicast.c | 268 ++++++++++++++------------------ 7 files changed, 224 insertions(+), 261 deletions(-) diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index 21ff51c..c415321 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #include @@ -1162,14 +1163,13 @@ osm_dump_mc_record(IN osm_log_t * const p_log, IN const ib_member_rec_t * const p_mcmr, IN const osm_log_level_t log_level) { - + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; if (osm_log_is_active(p_log, log_level)) { osm_log(p_log, log_level, "MCMember Record dump:\n" - "\t\t\t\tMGID....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tPortGid.................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\t\t\tMGID....................%s\n" + "\t\t\t\tPortGid.................%s\n" "\t\t\t\tqkey....................0x%X\n" "\t\t\t\tmlid....................0x%X\n" "\t\t\t\tmtu.....................0x%X\n" @@ -1181,10 +1181,10 @@ osm_dump_mc_record(IN osm_log_t * const p_log, "\t\t\t\tScopeState..............0x%X\n" "\t\t\t\tProxyJoin...............0x%X\n" "", - cl_ntoh64(p_mcmr->mgid.unicast.prefix), - cl_ntoh64(p_mcmr->mgid.unicast.interface_id), - cl_ntoh64(p_mcmr->port_gid.unicast.prefix), - cl_ntoh64(p_mcmr->port_gid.unicast.interface_id), + inet_ntop(AF_INET6, p_mcmr->mgid.raw, gid_str, + sizeof gid_str), + inet_ntop(AF_INET6, p_mcmr->port_gid.raw, gid_str2, + sizeof gid_str2), cl_ntoh32(p_mcmr->qkey), cl_ntoh16(p_mcmr->mlid), p_mcmr->mtu, @@ -1204,6 +1204,7 @@ osm_dump_service_record(IN osm_log_t * const p_log, IN const ib_service_record_t * const p_sr, IN const osm_log_level_t log_level) { + char gid_str[INET6_ADDRSTRLEN]; char buf_service_key[35]; char buf_service_name[65]; @@ -1231,8 +1232,7 @@ osm_dump_service_record(IN osm_log_t * const p_log, osm_log(p_log, log_level, "Service Record dump:\n" "\t\t\t\tServiceID...............0x%016" PRIx64 "\n" - "\t\t\t\tServiceGID..............0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" + "\t\t\t\tServiceGID..............%s\n" "\t\t\t\tServiceP_Key............0x%X\n" "\t\t\t\tServiceLease............0x%X\n" "\t\t\t\tServiceKey..............%s\n" @@ -1269,8 +1269,8 @@ osm_dump_service_record(IN osm_log_t * const p_log, "\t\t\t\tServiceData64.2.........0x%016" PRIx64 "\n" "", cl_ntoh64(p_sr->service_id), - cl_ntoh64(p_sr->service_gid.unicast.prefix), - cl_ntoh64(p_sr->service_gid.unicast.interface_id), + inet_ntop(AF_INET6, p_sr->service_gid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_sr->service_pkey), cl_ntoh32(p_sr->service_lease), buf_service_key, @@ -1376,6 +1376,8 @@ osm_dump_inform_info_record(IN osm_log_t * const p_log, IN const ib_inform_info_record_t * const p_iir, IN const osm_log_level_t log_level) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; uint32_t qpn; uint8_t resp_time_val; @@ -1389,12 +1391,10 @@ osm_dump_inform_info_record(IN osm_log_t * const p_log, osm_log(p_log, log_level, "InformInfo Record dump:\n" "\t\t\t\tRID\n" - "\t\t\t\tSubscriberGID...........0x%016" PRIx64 - " : " "0x%016" PRIx64 "\n" + "\t\t\t\tSubscriberGID...........%s\n" "\t\t\t\tSubscriberEnum..........0x%X\n" "\t\t\t\tInformInfo dump:\n" - "\t\t\t\tgid.....................0x%016" PRIx64 - " : 0x%016" PRIx64 "\n" + "\t\t\t\tgid.....................%s\n" "\t\t\t\tlid_range_begin.........%u\n" "\t\t\t\tlid_range_end...........%u\n" "\t\t\t\tis_generic..............0x%X\n" @@ -1404,14 +1404,11 @@ osm_dump_inform_info_record(IN osm_log_t * const p_log, "\t\t\t\tqpn.....................0x%06X\n" "\t\t\t\tresp_time_val...........0x%X\n" "\t\t\t\tnode_type...............0x%06X\n" "", - cl_ntoh64(p_iir->subscriber_gid.unicast.prefix), - cl_ntoh64(p_iir->subscriber_gid.unicast. - interface_id), + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, + gid_str, sizeof gid_str), cl_ntoh16(p_iir->subscriber_enum), - cl_ntoh64(p_iir->inform_info.gid.unicast. - prefix), - cl_ntoh64(p_iir->inform_info.gid.unicast. - interface_id), + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, + gid_str2, sizeof gid_str2), cl_ntoh16(p_iir->inform_info.lid_range_begin), cl_ntoh16(p_iir->inform_info.lid_range_end), p_iir->inform_info.is_generic, @@ -1427,12 +1424,10 @@ osm_dump_inform_info_record(IN osm_log_t * const p_log, osm_log(p_log, log_level, "InformInfo Record dump:\n" "\t\t\t\tRID\n" - "\t\t\t\tSubscriberGID...........0x%016" PRIx64 - " : " "0x%016" PRIx64 "\n" + "\t\t\t\tSubscriberGID...........%s\n" "\t\t\t\tSubscriberEnum..........0x%X\n" "\t\t\t\tInformInfo dump:\n" - "\t\t\t\tgid.....................0x%016" PRIx64 - " : 0x%016" PRIx64 "\n" + "\t\t\t\tgid.....................%s\n" "\t\t\t\tlid_range_begin.........%u\n" "\t\t\t\tlid_range_end...........%u\n" "\t\t\t\tis_generic..............0x%X\n" @@ -1442,14 +1437,11 @@ osm_dump_inform_info_record(IN osm_log_t * const p_log, "\t\t\t\tqpn.....................0x%06X\n" "\t\t\t\tresp_time_val...........0x%X\n" "\t\t\t\tvendor_id...............0x%06X\n" "", - cl_ntoh64(p_iir->subscriber_gid.unicast.prefix), - cl_ntoh64(p_iir->subscriber_gid.unicast. - interface_id), + inet_ntop(AF_INET6, p_iir->subscriber_gid.raw, + gid_str, sizeof gid_str), cl_ntoh16(p_iir->subscriber_enum), - cl_ntoh64(p_iir->inform_info.gid.unicast. - prefix), - cl_ntoh64(p_iir->inform_info.gid.unicast. - interface_id), + inet_ntop(AF_INET6, p_iir->inform_info.gid.raw, + gid_str2, sizeof gid_str2), cl_ntoh16(p_iir->inform_info.lid_range_begin), cl_ntoh16(p_iir->inform_info.lid_range_end), p_iir->inform_info.is_generic, diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index e6146f7..ad9ae35 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -44,6 +44,7 @@ #include #include +#include #include #include #include @@ -281,14 +282,15 @@ void osm_infr_remove_from_db(IN osm_subn_t * p_subn, IN osm_log_t * p_log, IN osm_infr_t * p_infr) { + char gid_str[INET6_ADDRSTRLEN]; OSM_LOG_ENTER(p_log); OSM_LOG(p_log, OSM_LOG_DEBUG, - "Removing InformInfo Subscribing GID:0x%016" PRIx64 " : 0x%016" - PRIx64 " Enum:0x%X from Database\n", - cl_ntoh64(p_infr->inform_record.subscriber_gid.unicast.prefix), - cl_ntoh64(p_infr->inform_record.subscriber_gid.unicast. - interface_id), p_infr->inform_record.subscriber_enum); + "Removing InformInfo Subscribing GID:%s" + " Enum:0x%X from Database\n", + inet_ntop(AF_INET6, p_infr->inform_record.subscriber_gid.raw, + gid_str, sizeof gid_str), + p_infr->inform_record.subscriber_enum); osm_dump_inform_info(p_log, &(p_infr->inform_record.inform_info), OSM_LOG_DEBUG); @@ -560,6 +562,7 @@ ib_api_status_t osm_report_notice(IN osm_log_t * const p_log, IN osm_subn_t * p_subn, IN ib_mad_notice_attr_t * p_ntc) { + char gid_str[INET6_ADDRSTRLEN]; osm_infr_match_ctxt_t context; cl_list_t infr_to_remove_list; osm_infr_t *p_infr_rec; @@ -582,25 +585,23 @@ osm_report_notice(IN osm_log_t * const p_log, if (ib_notice_is_generic(p_ntc)) { OSM_LOG(p_log, OSM_LOG_INFO, "Reporting Generic Notice type:%u num:%u" - " from LID:%u GID:0x%016" PRIx64 - ",0x%016" PRIx64 "\n", + " from LID:%u GID:%s\n", ib_notice_get_type(p_ntc), cl_ntoh16(p_ntc->g_or_v.generic.trap_num), cl_ntoh16(p_ntc->issuer_lid), - cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), - cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) + inet_ntop(AF_INET6, p_ntc->issuer_gid.raw, gid_str, + sizeof gid_str) ); } else { OSM_LOG(p_log, OSM_LOG_INFO, "Reporting Vendor Notice type:%u vend:%u dev:%u" - " from LID:%u GID:0x%016" PRIx64 - ",0x%016" PRIx64 "\n", + " from LID:%u GID:%s\n", ib_notice_get_type(p_ntc), cl_ntoh32(ib_notice_get_vend_id(p_ntc)), cl_ntoh16(p_ntc->g_or_v.vend.dev_id), cl_ntoh16(p_ntc->issuer_lid), - cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), - cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) + inet_ntop(AF_INET6, p_ntc->issuer_gid.raw, gid_str, + sizeof gid_str) ); } diff --git a/opensm/opensm/osm_perfmgr.c b/opensm/opensm/osm_perfmgr.c index 9892335..e283fa7 100644 --- a/opensm/opensm/osm_perfmgr.c +++ b/opensm/opensm/osm_perfmgr.c @@ -55,6 +55,7 @@ #include #include #include +#include #include #include #include @@ -1072,6 +1073,7 @@ osm_perfmgr_log_events(osm_perfmgr_t * pm, __monitored_node_t *mon_node, uint8_t **********************************************************************/ static void osm_pc_rcv_process(void *context, void *data) { + char gid_str[INET6_ADDRSTRLEN]; osm_perfmgr_t *const pm = (osm_perfmgr_t *) context; osm_madw_t *p_madw = (osm_madw_t *) data; osm_madw_context_t *mad_context = &(p_madw->context); @@ -1114,11 +1116,11 @@ static void osm_pc_rcv_process(void *context, void *data) OSM_LOG(pm->log, OSM_LOG_VERBOSE, "Redirection to LID %u " - "GID 0x%016" PRIx64 " : 0x%016" PRIx64 + "GID %s" " QP 0x%x received\n", cl_ntoh16(cpi->redir_lid), - cl_ntoh64(cpi->redir_gid.unicast.prefix), - cl_ntoh64(cpi->redir_gid.unicast.interface_id), + inet_ntop(AF_INET6, cpi->redir_gid.raw, gid_str, + sizeof gid_str), cl_ntoh32(cpi->redir_qp)); /* LID or GID redirection ? */ diff --git a/opensm/opensm/osm_sa_informinfo.c b/opensm/opensm/osm_sa_informinfo.c index 3bb2295..334d22e 100644 --- a/opensm/opensm/osm_sa_informinfo.c +++ b/opensm/opensm/osm_sa_informinfo.c @@ -46,6 +46,7 @@ #include #include +#include #include #include #include @@ -336,6 +337,7 @@ static void osm_infr_rcv_process_get_method(IN osm_sa_t * sa, IN osm_madw_t * const p_madw) { + char gid_str[INET6_ADDRSTRLEN]; ib_sa_mad_t *p_rcvd_mad; const ib_inform_info_record_t *p_rcvd_rec; cl_qlist_t rec_list; @@ -376,10 +378,10 @@ osm_infr_rcv_process_get_method(IN osm_sa_t * sa, context.p_req_physp = p_req_physp; OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Query Subscriber GID:0x%016" PRIx64 " : 0x%016" PRIx64 + "Query Subscriber GID:%s" "(%02X) Enum:0x%X(%02X)\n", - cl_ntoh64(p_rcvd_rec->subscriber_gid.unicast.prefix), - cl_ntoh64(p_rcvd_rec->subscriber_gid.unicast.interface_id), + inet_ntop(AF_INET6, p_rcvd_rec->subscriber_gid.raw, + gid_str, sizeof gid_str), (p_rcvd_mad->comp_mask & IB_IIR_COMPMASK_SUBSCRIBERGID) != 0, cl_ntoh16(p_rcvd_rec->subscriber_enum), (p_rcvd_mad->comp_mask & IB_IIR_COMPMASK_ENUM) != 0); diff --git a/opensm/opensm/osm_sa_mcmember_record.c b/opensm/opensm/osm_sa_mcmember_record.c index 086f3a5..871f4c6 100644 --- a/opensm/opensm/osm_sa_mcmember_record.c +++ b/opensm/opensm/osm_sa_mcmember_record.c @@ -47,6 +47,7 @@ #include #include +#include #include #include #include @@ -830,6 +831,7 @@ osm_mcmr_rcv_create_new_mgrp(IN osm_sa_t * sa, IN const osm_physp_t * const p_physp, OUT osm_mgrp_t ** pp_mgrp) { + char gid_str[INET6_ADDRSTRLEN]; ib_net16_t mlid; uint8_t zero_mgid, valid; uint8_t scope, i; @@ -891,10 +893,9 @@ osm_mcmr_rcv_create_new_mgrp(IN osm_sa_t * sa, memcpy(&p_mgid->raw[10], &mlid, sizeof(uint16_t)); memcpy(&p_mgid->raw[12], &mlid, sizeof(uint16_t)); OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Allocated new MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n", - cl_ntoh64(p_mgid->unicast.prefix), - cl_ntoh64(p_mgid->unicast.interface_id)); + "Allocated new MGID:%s\n", + inet_ntop(AF_INET6, p_mgid->raw, gid_str, + sizeof gid_str)); } else { /* a specific MGID was requested so validate the resulting MGID */ valid = __validate_requested_mgid(sa, &mcm_rec); @@ -978,6 +979,7 @@ typedef struct osm_sa_pr_mcmr_search_ctxt { static void __search_mgrp_by_mgid(IN osm_mgrp_t * const p_mgrp, IN void *context) { + char gid_str[INET6_ADDRSTRLEN]; osm_sa_pr_mcmr_search_ctxt_t *p_ctxt = (osm_sa_pr_mcmr_search_ctxt_t *) context; const ib_gid_t *p_recvd_mgid; @@ -1017,9 +1019,9 @@ __search_mgrp_by_mgid(IN osm_mgrp_t * const p_mgrp, IN void *context) (rcv_interface_id & INT_ID_MASK)) { OSM_LOG(sa->p_log, OSM_LOG_INFO, "Special Case Solicited Node Mcast " - "Join for MGID 0x%016" PRIx64 - " : 0x%016" PRIx64 "\n", - rcv_prefix, rcv_interface_id); + "Join for MGID %s\n", + inet_ntop(AF_INET6, p_recvd_mgid->raw, gid_str, + sizeof gid_str)); } else return; } else @@ -1028,10 +1030,9 @@ __search_mgrp_by_mgid(IN osm_mgrp_t * const p_mgrp, IN void *context) if (p_ctxt->p_mgrp) { OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B30: " - "Multiple MC groups for MGID " - "0x%016" PRIx64 " : 0x%016" PRIx64 "\n", - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast.prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast.interface_id)); + "Multiple MC groups for MGID %s\n", + inet_ntop(AF_INET6, p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str)); return; } p_ctxt->p_mgrp = p_mgrp; @@ -1093,6 +1094,8 @@ static void __osm_mcmr_rcv_leave_mgrp(IN osm_sa_t * sa, IN osm_madw_t * const p_madw) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; boolean_t valid; osm_mgrp_t *p_mgrp; ib_api_status_t status; @@ -1176,18 +1179,12 @@ __osm_mcmr_rcv_leave_mgrp(IN osm_sa_t * sa, CL_PLOCK_RELEASE(sa->p_lock); OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B25: " "Received an invalid delete request for " - "MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 " for " - "PortGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n", - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - prefix), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - interface_id), - cl_ntoh64(p_recvd_mcmember_rec->port_gid. - unicast.prefix), - cl_ntoh64(p_recvd_mcmember_rec->port_gid. - unicast.interface_id)); + "MGID: %s for PortGID: %s\n", + inet_ntop(AF_INET6, p_recvd_mcmember_rec->mgid.raw, + gid_str, sizeof gid_str), + inet_ntop(AF_INET6, + p_recvd_mcmember_rec->port_gid.raw, + gid_str2, sizeof gid_str2)); osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_REQ_INVALID); goto Exit; @@ -1195,10 +1192,9 @@ __osm_mcmr_rcv_leave_mgrp(IN osm_sa_t * sa, } else { CL_PLOCK_RELEASE(sa->p_lock); OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Failed since multicast group 0x%16" - PRIx64 " : 0x%016" PRIx64 " not present\n", - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast.interface_id)); + "Failed since multicast group %s not present\n", + inet_ntop(AF_INET6, p_recvd_mcmember_rec->mgid.raw, + gid_str, sizeof gid_str)); osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_REQ_INVALID); goto Exit; } @@ -1217,6 +1213,7 @@ static void __osm_mcmr_rcv_join_mgrp(IN osm_sa_t * sa, IN osm_madw_t * const p_madw) { + char gid_str[INET6_ADDRSTRLEN]; boolean_t valid; osm_mgrp_t *p_mgrp = NULL; ib_api_status_t status; @@ -1296,13 +1293,12 @@ __osm_mcmr_rcv_join_mgrp(IN osm_sa_t * sa, OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B10: " "Provided Join State != FullMember - " "required for create, " - "MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 " from port 0x%016" PRIx64 + "MGID: %s from port 0x%016" PRIx64 " (%s)\n", - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - prefix), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - interface_id), cl_ntoh64(portguid), + inet_ntop(AF_INET6, + p_recvd_mcmember_rec->mgid.raw, + gid_str, sizeof gid_str), + cl_ntoh64(portguid), p_port->p_node->print_desc); osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_REQ_INVALID); @@ -1332,16 +1328,15 @@ __osm_mcmr_rcv_join_mgrp(IN osm_sa_t * sa, "method = %s, scope_state = 0x%x, " "component mask = 0x%016" PRIx64 ", " "expected comp mask = 0x%016" PRIx64 ", " - "MGID: 0x%016" PRIx64 " : 0x%016" PRIx64 - " from port 0x%016" PRIx64 " (%s)\n", + "MGID: %s from port 0x%016" PRIx64 " (%s)\n", ib_get_sa_method_str(p_sa_mad->method), p_recvd_mcmember_rec->scope_state, cl_ntoh64(p_sa_mad->comp_mask), CL_NTOH64(REQUIRED_MC_CREATE_COMP_MASK), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - prefix), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - interface_id), cl_ntoh64(portguid), + inet_ntop(AF_INET6, + p_recvd_mcmember_rec->mgid.raw, + gid_str, sizeof gid_str), + cl_ntoh64(portguid), p_port->p_node->print_desc); osm_sa_send_error(sa, p_madw, @@ -1534,6 +1529,7 @@ static void __osm_sa_mcm_by_comp_mask_cb(IN osm_mgrp_t * const p_mgrp, IN void *context) { + char gid_str[INET6_ADDRSTRLEN]; osm_sa_mcmr_search_ctxt_t *const p_ctxt = (osm_sa_mcmr_search_ctxt_t *) context; osm_sa_t *sa = p_ctxt->sa; @@ -1671,13 +1667,10 @@ __osm_sa_mcm_by_comp_mask_cb(IN osm_mgrp_t * const p_mgrp, &(p_mcm_port->port_gid), sizeof(ib_gid_t)); OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Record of port_gid: 0x%016" PRIx64 - "0x%016" PRIx64 + "Record of port_gid: %s" " in multicast_lid: 0x%X is returned\n", - cl_ntoh64(match_rec.port_gid.unicast. - prefix), - cl_ntoh64(match_rec.port_gid.unicast. - interface_id), + inet_ntop(AF_INET6, match_rec.port_gid.raw, + gid_str, sizeof gid_str), cl_ntoh16(p_mgrp->mlid)); match_rec.proxy_join = @@ -1797,6 +1790,8 @@ Exit: **********************************************************************/ void osm_mcmr_rcv_process(IN void *context, IN void *data) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; osm_sa_t *sa = context; osm_madw_t *p_madw = data; ib_sa_mad_t *p_sa_mad; @@ -1822,18 +1817,15 @@ void osm_mcmr_rcv_process(IN void *context, IN void *data) OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B18: " "component mask = 0x%016" PRIx64 ", " "expected comp mask = 0x%016" PRIx64 ", " - "MGID: 0x%016" PRIx64 " : 0x%016" PRIx64 - " for PortGID: 0x%016" PRIx64 " : 0x%016" - PRIx64 "\n", cl_ntoh64(p_sa_mad->comp_mask), + "MGID: %s for PortGID: %s\n", + cl_ntoh64(p_sa_mad->comp_mask), CL_NTOH64(JOIN_MC_COMP_MASK), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - prefix), - cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. - interface_id), - cl_ntoh64(p_recvd_mcmember_rec->port_gid. - unicast.prefix), - cl_ntoh64(p_recvd_mcmember_rec->port_gid. - unicast.interface_id)); + inet_ntop(AF_INET6, + p_recvd_mcmember_rec->mgid.raw, + gid_str, sizeof gid_str), + inet_ntop(AF_INET6, + p_recvd_mcmember_rec->port_gid.raw, + gid_str2, sizeof gid_str2)); osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_REQ_INVALID); diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index f863735..4fe491c 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -46,6 +46,7 @@ #endif /* HAVE_CONFIG_H */ #include +#include #include #include #include @@ -1474,6 +1475,7 @@ static void __osm_pr_get_mgrp(IN osm_sa_t * sa, IN const osm_madw_t * const p_madw, OUT osm_mgrp_t ** pp_mgrp) { + char gid_str[INET6_ADDRSTRLEN]; ib_path_rec_t *p_pr; const ib_sa_mad_t *p_sa_mad; ib_net64_t comp_mask; @@ -1491,9 +1493,9 @@ __osm_pr_get_mgrp(IN osm_sa_t * sa, if (status != IB_SUCCESS) { OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F09: " "No MC group found for PathRecord destination " - "GID 0x%016" PRIx64 " : " "0x%016" PRIx64 "\n", - cl_ntoh64(p_pr->dgid.unicast.prefix), - cl_ntoh64(p_pr->dgid.unicast.interface_id)); + "GID %s\n", + inet_ntop(AF_INET6, p_pr->dgid.raw, gid_str, + sizeof gid_str)); goto Exit; } } diff --git a/opensm/osmtest/osmt_multicast.c b/opensm/osmtest/osmt_multicast.c index 15b656f..e1ca3e5 100644 --- a/opensm/osmtest/osmt_multicast.c +++ b/opensm/osmtest/osmt_multicast.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #include @@ -149,6 +150,7 @@ __match_mgids(IN const void *const p_object, IN void *context) ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) { + char gid_str[INET6_ADDRSTRLEN]; ib_api_status_t status = IB_SUCCESS; osmv_user_query_t user; osmv_query_req_t req; @@ -240,10 +242,9 @@ ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) if (p_mgids_res != cl_list_end(p_mgids_list)) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_query_mcast: ERR 0265: " - "MCG MGIDs are the same - invalid MGID : 0x%016" - PRIx64 " 0x%016" PRIx64 "\n", - cl_ntoh64(p_rec->mgid.unicast.prefix), - cl_ntoh64(p_rec->mgid.unicast.interface_id)); + "MCG MGIDs are the same - invalid MGID : %s\n", + inet_ntop(AF_INET6, p_rec->mgid.raw, gid_str, + sizeof gid_str)); status = IB_ERROR; goto Exit; @@ -477,6 +478,8 @@ osmt_init_mc_query_rec(IN osmtest_t * const p_osmt, ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) { + char gid_str[INET6_ADDRSTRLEN]; + char gid_str2[INET6_ADDRSTRLEN]; ib_api_status_t status; ib_member_rec_t mc_req_rec; ib_member_rec_t *p_mc_res; @@ -595,12 +598,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) } else { osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Non-IPoIB MC Groups exist: mgid=0x%016" PRIx64 - ":0x%016" PRIx64 "\n", - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. - prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. - interface_id)); + "Non-IPoIB MC Groups exist: mgid=%s\n", + inet_ntop(AF_INET6, + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str)); mcg_outside_test_cnt++; } @@ -635,13 +636,13 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) "Joining an existing IPoIB multicast group\n"); osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Sent Join request with :\n\t\tport_gid=0x%016" PRIx64 - ":0x%016" PRIx64 ", mgid=0x%016" PRIx64 ":0x%016" PRIx64 - "\n\t\tjoin state= 0x%x, response is : %s\n", - cl_ntoh64(mc_req_rec.port_gid.unicast.prefix), - cl_ntoh64(mc_req_rec.port_gid.unicast.interface_id), - cl_ntoh64(mc_req_rec.mgid.unicast.prefix), - cl_ntoh64(mc_req_rec.mgid.unicast.interface_id), + "Sent Join request with :\n\t\tport_gid=%s" + ", mgid=%s\n" + "\t\tjoin state= 0x%x, response is : %s\n", + inet_ntop(AF_INET6, mc_req_rec.port_gid.raw, + gid_str, sizeof gid_str), + inet_ntop(AF_INET6, mc_req_rec.mgid.raw, + gid_str2, sizeof gid_str2), (mc_req_rec.scope_state & 0x0F), ib_get_err_str(status)); if (status != IB_SUCCESS) { @@ -699,11 +700,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (status != IB_SUCCESS) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 02EF: " - "Query as Full Member of already existing ipoib group 0x%016" - PRIx64 ":0x%016" PRIx64 " has failed\n", - cl_ntoh64(mc_req_rec.mgid.unicast.prefix), - cl_ntoh64(mc_req_rec.mgid.unicast. - interface_id)); + "Query as Full Member of already existing " + "ipoib group gid %s has failed\n", + inet_ntop(AF_INET6, mc_req_rec.mgid.raw, + gid_str, sizeof gid_str)); goto Exit; } @@ -1382,10 +1382,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1414,10 +1413,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1445,10 +1443,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1475,10 +1472,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1516,10 +1512,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1552,10 +1547,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1588,10 +1582,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1610,9 +1603,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) (p_mc_res->mgid.multicast.raw_group_id[1] != 0x1B)) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 0209: " - "Validating MGID failed. MGID:0x%016" PRIx64 ":%016" - PRIx64 "\n", cl_ntoh64(p_mc_res->mgid.unicast.prefix), - cl_ntoh64(p_mc_res->mgid.unicast.interface_id) + "Validating MGID failed. MGID:%s\n", + inet_ntop(AF_INET6, p_mc_res->mgid.raw, gid_str, + sizeof gid_str) ); status = IB_ERROR; goto Exit; @@ -1656,10 +1649,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1671,10 +1663,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Checking Create given valid MGID=0x%016" PRIx64 " : " - "0x%016" PRIx64 " (o15.0.1.6)...\n", - cl_ntoh64(mc_req_rec.mgid.unicast.prefix), - cl_ntoh64(mc_req_rec.mgid.unicast.interface_id)); + "Checking Create given valid MGID=%s (o15.0.1.6)...\n", + inet_ntop(AF_INET6, mc_req_rec.mgid.raw, gid_str, + sizeof gid_str)); /* Before creation, need to check that this group doesn't exist */ osm_log(&p_osmt->log, OSM_LOG_INFO, @@ -1707,20 +1698,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Now creating group with given valid MGID=0x%016" PRIx64 " : " - "0x%016" PRIx64 " (o15.0.1.6)...\n", - cl_ntoh64(mc_req_rec.mgid.unicast.prefix), - cl_ntoh64(mc_req_rec.mgid.unicast.interface_id)); + "Now creating group with given valid MGID=%s (o15.0.1.6)...\n", + inet_ntop(AF_INET6, mc_req_rec.mgid.raw, gid_str, + sizeof gid_str)); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 0211: " - "Failed to create MCG for MGID=0x%016" PRIx64 " : " - "0x%016" PRIx64 " (o15.0.1.6) - got %s/%s\n", - cl_ntoh64(good_mgid.unicast.prefix), - cl_ntoh64(good_mgid.unicast.interface_id), + "Failed to create MCG for MGID=%s (o15.0.1.6) - got %s/%s\n", + inet_ntop(AF_INET6, good_mgid.raw, gid_str, + sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; @@ -1730,10 +1719,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1747,9 +1735,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) (p_mc_res->mgid.multicast.raw_group_id[1] != 0x1C)) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 0212: " - "Validating MGID failed. MGID:0x%016" PRIx64 ":%016" - PRIx64 "\n", cl_ntoh64(p_mc_res->mgid.unicast.prefix), - cl_ntoh64(p_mc_res->mgid.unicast.interface_id) + "Validating MGID failed. MGID:%s\n", + inet_ntop(AF_INET6, p_mc_res->mgid.raw, gid_str, + sizeof gid_str) ); status = IB_ERROR; goto Exit; @@ -1842,10 +1830,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Checking local scope with full member \n\t\tand valid mgid 0x%016" - PRIx64 ":0x%016" PRIx64 " ... (o15.0.1.6)...\n", - cl_ntoh64(mc_req_rec.mgid.unicast.prefix), - cl_ntoh64(mc_req_rec.mgid.unicast.interface_id)); + "Checking local scope with full member \n\t\tand valid mgid %s" + " ... (o15.0.1.6)...\n", + inet_ntop(AF_INET6, mc_req_rec.mgid.raw, gid_str, + sizeof gid_str)); mc_req_rec.mgid = good_mgid; @@ -1856,10 +1844,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (status != IB_SUCCESS) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 0216: " - "Failed to create MCG for MGID=0x%016" PRIx64 " : " - "0x%016" PRIx64 " - got %s/%s\n", - cl_ntoh64(good_mgid.unicast.prefix), - cl_ntoh64(good_mgid.unicast.interface_id), + "Failed to create MCG for MGID=%s - got %s/%s\n", + inet_ntop(AF_INET6, good_mgid.raw, gid_str, + sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; @@ -1869,10 +1856,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1932,10 +1918,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1999,10 +1984,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -2065,10 +2049,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast.prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast.interface_id), + "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -2476,10 +2459,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* Prepare another MCG for the following tests : */ osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Checking Create given MGID=0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n\t\t(o15.0.1.4)...\n", - cl_ntoh64(osm_ipoib_mgid.unicast.prefix), - cl_ntoh64(osm_ipoib_mgid.unicast.interface_id)); + "Checking Create given MGID=%s\n\t\t(o15.0.1.4)...\n", + inet_ntop(AF_INET6, osm_ipoib_mgid.raw, gid_str, + sizeof gid_str)); mc_req_rec.mgid = good_mgid; mc_req_rec.mgid.raw[12] = 0xAA; @@ -2493,10 +2475,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (status != IB_SUCCESS) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 02BE: " - "Failed to create MCG for 0x%016" PRIx64 " : " - "0x%016" PRIx64 " - got %s/%s\n", - cl_ntoh64(good_mgid.unicast.prefix), - cl_ntoh64(good_mgid.unicast.interface_id), + "Failed to create MCG for %s - got %s/%s\n", + inet_ntop(AF_INET6, good_mgid.raw, gid_str, + sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; @@ -2706,12 +2687,11 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 02A9: " "Successful deletion of remote port guid with local one MGID : " - "0x%016" PRIx64 " : 0x%016" PRIx64 - ", Got : %s/%s\n", - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. - prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. - interface_id), ib_get_err_str(status), + "%s, Got : %s/%s\n", + inet_ntop(AF_INET6, + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str), + ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; @@ -2737,12 +2717,11 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 02B0: " "Failed to delete mgid with remote port guid MGID : " - "0x%016" PRIx64 " : 0x%016" PRIx64 - ", Got : %s/%s\n", - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. - prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. - interface_id), ib_get_err_str(status), + "%s, Got : %s/%s\n", + inet_ntop(AF_INET6, + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str), + ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; @@ -2816,12 +2795,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) ib_sa_mad_get_payload_ptr(&res_sa_mad); osm_log(&p_osmt->log, OSM_LOG_VERBOSE, "osmt_run_mcast_flow : " - "Created MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 " MLID:0x%04X\n", - cl_ntoh64(p_recvd_rec->mgid.unicast. - prefix), - cl_ntoh64(p_recvd_rec->mgid.unicast. - interface_id), + "Created MGID:%s MLID:0x%04X\n", + inet_ntop(AF_INET6, + p_recvd_rec->mgid.raw, + gid_str, sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), @@ -2881,10 +2858,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osm_log(&p_osmt->log, OSM_LOG_VERBOSE, "osmt_run_mcast_flow : " - "Sending request to delete MGID : 0x%016" PRIx64 - " : 0x%016" PRIx64 ", scope_state : 0x%02X\n", - cl_ntoh64(mc_req_rec.mgid.unicast.prefix), - cl_ntoh64(mc_req_rec.mgid.unicast.interface_id), + "Sending request to delete MGID : %s" + ", scope_state : 0x%02X\n", + inet_ntop(AF_INET6, mc_req_rec.mgid.raw, + gid_str, sizeof gid_str), mc_req_rec.scope_state); status = osmt_send_mcast_request(p_osmt, 0, /* delete flag */ &mc_req_rec, @@ -2893,13 +2870,11 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (status != IB_SUCCESS) { osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 02FF: " - "Failed to delete MGID : 0x%016" PRIx64 - " : 0x%016" PRIx64 + "Failed to delete MGID : %s" " ,\n\t\t it is not our MCG, Status : %s/%s\n", - cl_ntoh64(p_mgrp->mcmember_rec.mgid. - unicast.prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid. - unicast.interface_id), + inet_ntop(AF_INET6, + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) @@ -2963,26 +2938,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* This means that we still have an mgrp that we created!! */ osm_log(&p_osmt->log, OSM_LOG_ERROR, "osmt_run_mcast_flow: ERR 02FE: " - "Wasn't able to erase mgrp with MGID:0x%016" - PRIx64 " : 0x%016" PRIx64 + "Wasn't able to erase mgrp with MGID:%s" " MLID:0x%04X\n", - cl_ntoh64(p_mgrp->mcmember_rec.mgid. - unicast.prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid. - unicast.interface_id), mlid); + inet_ntop(AF_INET6, + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str), + mlid); got_error = TRUE; } else { osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " - "Still exists %s MGID:0x%016" PRIx64 - " : 0x%016" PRIx64 "\n", + "Still exists %s MGID:%s\n", (IS_IPOIB_MGID (&p_mgrp->mcmember_rec. mgid)) ? "IPoIB" : "non-IPoIB", - cl_ntoh64(p_mgrp->mcmember_rec.mgid. - unicast.prefix), - cl_ntoh64(p_mgrp->mcmember_rec.mgid. - unicast.interface_id)); + inet_ntop(AF_INET6, + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str)); } p_mgrp = (osmtest_mgrp_t *) cl_qmap_next(&p_mgrp->map_item); -- 1.5.4.5 From hrosenstock at xsigo.com Thu Jul 10 18:26:56 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 18:26:56 -0700 Subject: [ofa-general] Cisco 7008 Internal/External Port Mappings In-Reply-To: References: <20080710110157.GA4004@mtls03> <20080710123304.GD4004@mtls03> <5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu> <1215725788.18371.193.camel@hrosenstock-ws.xsigo.com> Message-ID: <1215739616.18371.196.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-07-10 at 19:15 -0400, Charles Taylor wrote: > Nope. Looks pretty much the same as ibnetdiscover without the "- > g". It would be really useful if it *did* work. :) IMO it should be made to work properly. > Thanks for the suggestion though. I think we just may be stuck > mapping it empirically. For now :-( > I'm sure the pattern will become obvious > quickly enough. I'll be curious about your findings (and the ibnetdiscover file). -- Hal > Charlie Taylor > UF HPC Center > > On Jul 10, 2008, at 5:36 PM, Hal Rosenstock wrote: > > > On Thu, 2008-07-10 at 17:27 -0400, Charles Taylor wrote: > >> We've got a couple of Cisco 7008/Topspin 270 core switches. Much > >> of > >> the diagnostic and topology information we get from the ib diag tools > >> in OFED refer to the internal ports rather than the actual external > >> slot/port. > > > > Does ibnetdiscover -g work ? There was some work done to do this > > mapping > > for Cisco but it may be out of date. > > > > -- Hal > > > >> We could deduce the mappings, I suppose, by tracing > >> cables and using ibtracert between nodes but that seems like a real > >> pain. I was wondering if anyone on this list knows how that > >> mapping > >> works. The chassis itself consists of 8 "ib port" blades in the > >> front labelled as slots 1 - 8. There are then six "fabric > >> controller" slots in the back of the chassis labelled 9 - 14. > >> Slots 11 and 12 are "special" as they hold the "master" and "standby > >> master" fabric controllers. > >> > >> Seems like the FC's in slots 11 and 12 connect the the FCs in slots > >> 9,10,13,14 which in turn connect to the external port blades in slots > >> 1 - 8. > >> > >> Again, all of the diagnostic information refers to the internal FC > >> ports and mapping those to the external ports (to which we actually > >> connect leaf switches) seems non-trivial. I think there was even a > >> document somewhere that described the mapping. We just can't find > >> it > >> now. > >> > >> Thanks, > >> > >> Charlie Taylor > >> UF HPC Center > >> _______________________________________________ > >> general mailing list > >> general at lists.openfabrics.org > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hal.rosenstock at gmail.com Thu Jul 10 19:14:05 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 22:14:05 -0400 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: References: <1215725123.18371.185.camel@hrosenstock-ws.xsigo.com> Message-ID: On Thu, Jul 10, 2008 at 7:47 PM, Yicheng Jia wrote: > >> You would also need to walk the ports to active (via MADs) after setting >> the LIDs as well as setup the switch forwarding table (LFT). > > Are these done by opensm? Yes. > Are these only two tasks that need to be done by > opensm in a single unmanaged switch network? No; I think another task was mentioned in Jim's minimalist approach email and there are a few others. Just using OpenSM is much simpler IMO unless you want to invest some work in learning what needs to be done and how to do it. -- Hal > Best, > Yicheng > > > > Hal Rosenstock > > 07/10/2008 04:39 PM > > To > Jim Mott > cc > Yicheng Jia , general at lists.openfabrics.org > Subject > RE: [ofa-general] minimum sw components requirement for driver/opensm > in a single unmanaged switch network > > > > > On Thu, 2008-07-10 at 14:17 -0700, Jim Mott wrote: >> If you want to avoid all the SM stuff, and are willing to program the >> switches directly (a few mads), > > You would also need to walk the ports to active (via MADs) after setting > the LIDs as well as setup the switch forwarding table (LFT). > > -- Hal > >> then I've used schemes like: >> >> >> >> Node LID=base + (switch port * constant) (base=0, constant = 1 works) >> >> >> >> Then to figure out QP connections, you just use a function of 3 >> parameters: >> >> my_qp_num = fn_sqp(my_node, target_node, qp_num) >> >> target_qp_num = fn_tqp(my_node, target_node, qp_num) >> >> Where qp_num is a small number between 0 and the maximum number of QPs >> you need active between any 2 endpoints. >> >> >> >> With the above scheme, you know your node_id (switch port number), >> your lid, the lid of the target node, and the QPs on both sides. >> From there on, it is clear sailing. You don't even need to send MADs; >> just transition the QP up and try and use it. If it works, you are >> done. If not, reset, up, wait for him to connect and send something >> to you. A little timer to make sure everybody retries once in awhile >> and what can go wrong? >> >> >> >> Jim >> >> From:general-bounces at lists.openfabrics.org [mailto:general- >> bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia >> Sent: Thursday, July 10, 2008 2:59 PM >> To: general at lists.openfabrics.org >> Subject: [ofa-general] minimum sw components requirement for >> driver/opensm in a single unmanaged switch network >> >> >> >> >> >> Hi Folks, >> >> I have a IB network which consists of only a single unmanaged switch, >> all end nodes connecting with the switch only need to do RDMA >> read/write operation with each other. My question is, what are the >> indispensable modules in driver's core and opensm that make the >> network up and run? >> >> I've been using only ib_mad module in driver's core with a managed >> switch before, and the network works fine. So I assume that only the >> ib_mad module in driver's core and SM in opensm are mandatory in my >> network. The LIDs are assigned by them. The SA and CM modules are not >> useful in my case. Am I right? >> >> I need to minimize driver and opensm to fit them in my network, the >> HCA driver is mthca. >> >> Best, >> Yicheng >> >> _____________________________________________________________________________ >> Scanned by IBM Email Security Management Services powered by >> MessageLabs. For more information please visit http://www.ers.ibm.com >> >> _____________________________________________________________________________ >> >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general > > > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by MessageLabs. > For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by MessageLabs. > For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From hal.rosenstock at gmail.com Thu Jul 10 19:16:23 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 10 Jul 2008 22:16:23 -0400 Subject: [ofa-general] minimum sw components requirement for driver/opensm in a single unmanaged switch network In-Reply-To: References: Message-ID: On Thu, Jul 10, 2008 at 7:39 PM, Yicheng Jia wrote: > >> If you want to avoid all the SM stuff, and are willing to program the >> switches directly (a few mads) > > Is it done by opensm? Yes. > What information should be set up in the switch by > opensm? Things like the PortInfos and LFT. See IBA spec vol 1 14.2.5 >> Then to figure out QP connections, you just use a function of 3 >> parameters: >> my_qp_num = fn_sqp(my_node, target_node, qp_num) >> target_qp_num = fn_tqp(my_node, target_node, qp_num) >> Where qp_num is a small number between 0 and the maximum number of QPs you >> need active between any 2 endpoints. > > Can the qp_num be manually assigned? > Does it need opensm be involved? SM has nothing to do with QP numbers. >> If it works, you are done. If not, reset, up, wait for him to connect and >> send something to you. > > Is it reliable? I mean the QPs connection will keep alive during the QPs > lifecycle? For one thing, SM needs to try to keep ports at active. -- Hal > Best, > Yicheng > > > > "Jim Mott" > > 07/10/2008 04:17 PM > > To > "Yicheng Jia" , > cc > Subject > RE: [ofa-general] minimum sw components requirement for driver/opensm in a > single unmanaged switch network > > > > > If you want to avoid all the SM stuff, and are willing to program the > switches directly (a few mads), then I've used schemes like: > > Node LID=base + (switch port * constant) (base=0, constant = 1 works) > > Then to figure out QP connections, you just use a function of 3 parameters: > my_qp_num = fn_sqp(my_node, target_node, qp_num) > target_qp_num = fn_tqp(my_node, target_node, qp_num) > Where qp_num is a small number between 0 and the maximum number of QPs you > need active between any 2 endpoints. > > With the above scheme, you know your node_id (switch port number), your lid, > the lid of the target node, and the QPs on both sides. From there on, it > is clear sailing. You don't even need to send MADs; just transition the QP > up and try and use it. If it works, you are done. If not, reset, up, wait > for him to connect and send something to you. A little timer to make sure > everybody retries once in awhile and what can go wrong? > > Jim > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Yicheng Jia > Sent: Thursday, July 10, 2008 2:59 PM > To: general at lists.openfabrics.org > Subject: [ofa-general] minimum sw components requirement for driver/opensm > in a single unmanaged switch network > > > Hi Folks, > > I have a IB network which consists of only a single unmanaged switch, all > end nodes connecting with the switch only need to do RDMA read/write > operation with each other. My question is, what are the indispensable > modules in driver's core and opensm that make the network up and run? > > I've been using only ib_mad module in driver's core with a managed switch > before, and the network works fine. So I assume that only the ib_mad module > in driver's core and SM in opensm are mandatory in my network. The LIDs are > assigned by them. The SA and CM modules are not useful in my case. Am I > right? > > I need to minimize driver and opensm to fit them in my network, the HCA > driver is mthca. > > Best, > Yicheng > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by MessageLabs. > For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by MessageLabs. > For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > _____________________________________________________________________________ > Scanned by IBM Email Security Management Services powered by MessageLabs. > For more information please visit http://www.ers.ibm.com > _____________________________________________________________________________ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From vlad at lists.openfabrics.org Fri Jul 11 02:39:50 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Fri, 11 Jul 2008 02:39:50 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080711-0200 daily build status Message-ID: <20080711093950.1830BE2800E@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080711-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ossrosch at linux.vnet.ibm.com Fri Jul 11 02:49:42 2008 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Fri, 11 Jul 2008 11:49:42 +0200 Subject: [ofa-general] tarfiles for libehca, libsdp? In-Reply-To: <20080709165522.9760B1C8E49@ece06.nas.nasa.gov> References: <20080709165522.9760B1C8E49@ece06.nas.nasa.gov> Message-ID: <200807111149.43036.ossrosch@linux.vnet.ibm.com> Hi Bryan, > >From where can I download the tar files for libehca and libsdp? They are > not available under the http://www.openfabrics.org/downloads/ directory. For the libehca please download the http://www.openfabrics.org/downloads/OFED/ofed-1.3/OFED-1.3.tgz file. In OFED-1.3/SRPMS/ directory you will find the src rpm for libehca. regards Stefan From 7dlenz at ycedc.org Fri Jul 11 04:40:09 2008 From: 7dlenz at ycedc.org (jayme pat) Date: Fri, 11 Jul 2008 11:40:09 +0000 Subject: [ofa-general] to general Message-ID: <000401c8e359$06190f4c$0edcb9b1@ebhljei> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.rahwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From rdreier at cisco.com Fri Jul 11 08:33:53 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 11 Jul 2008 08:33:53 -0700 Subject: [ofa-general][PATCH 8/11 v3] mlx4: Default value for automatic completion vector selection In-Reply-To: <4874BEA9.4010202@mellanox.co.il> (Yevgeny Petrilin's message of "Wed, 09 Jul 2008 16:35:37 +0300") References: <4874BEA9.4010202@mellanox.co.il> Message-ID: I haven't read this all very carefully yet, but this seems funny: > IB_CQ_VECTOR_LEAST_ATTACHED is redefined in device.h as MLX4_ANY_VECTOR > because we don't want all mlx4_core clients (Ethernet and FCoE) to > include OK, but... > --- a/include/linux/mlx4/device.h > +++ b/include/linux/mlx4/device.h > @@ -37,6 +37,8 @@ > #include > #include > > +#include > > +#define MLX4_ANY_VECTOR IB_CQ_VECTOR_LEAST_ATTACHED you just add to a common header, which means all mlx4_core clients end up including it! I would suggest just doing #define MLX4_LEAST_ATTACHED_VECTOR 0xffffffff and then doing something like this in the IB driver: err = mlx4_cq_alloc(dev->dev, entries, &cq->buf.mtt, uar, cq->db.dma, &cq->mcq, vector == IB_CQ_VECTOR_LEAST_ATTACHED ? MLX4_LEAST_ATTACHED_VECTOR : vector, 0); > +#define IB_CQ_VECTOR_LEAST_ATTACHED 0xff And making this such a low value seems like a bad idea, given that even mlx4 devices have a 256 entry MSI-X table, and in the not-too distant future even bigger tables are easily possible. - R. From debtremove at gmx.com Fri Jul 11 09:16:39 2008 From: debtremove at gmx.com (Debt Remove) Date: Fri, 11 Jul 2008 09:16:39 -0700 Subject: [ofa-general] Stop The Calls Message-ID: An HTML attachment was scrubbed... URL: From tewilcox at cisco.com Fri Jul 11 09:35:51 2008 From: tewilcox at cisco.com (Ted Wilcox (tewilcox)) Date: Fri, 11 Jul 2008 09:35:51 -0700 Subject: [ofa-general] Cisco 7008 Internal/External Port Mappings In-Reply-To: References: <20080710110157.GA4004@mtls03><20080710123304.GD4004@mtls03><5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu><1215725788.18371.193.camel@hrosenstock-ws.xsigo.com> Message-ID: Attached is a text file (unix file format) with the port mappings. As you surmised, the leaf chips are in slots 9, 10, 13, 14, and the spines are in 11, 12. Our nomenclature is that the leaf cards are "Node" cards (Node 1 = Slot 9, Node 2 = Slot 10, etc.) and the spine cards are "Core" cards (Core 1 = slot 11, Core 2 = slot 12). I apologize for the somewhat goofy naming; it came from the early hardware design. If you're running the Cisco SM, we have recently released a pretty general-purpose network debugging/maintenance tool called FACT which may help you out (http://tools.cisco.com/support/downloads/go/Redirect.x?mdfid=279966112 -- look under "Server Fabric Software"). It requires a cisco.com login, but not a support contract. -Ted. P.S. I don't generally keep up with ofa-general (my apologies), so please cc' me on any replies, flames, etc. -----Original Message----- From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Charles Taylor Sent: Thursday, July 10, 2008 4:15 PM To: general-list Subject: Re: [ofa-general] Cisco 7008 Internal/External Port Mappings Nope. Looks pretty much the same as ibnetdiscover without the "- g". It would be really useful if it *did* work. :) Thanks for the suggestion though. I think we just may be stuck mapping it empirically. I'm sure the pattern will become obvious quickly enough. Charlie Taylor UF HPC Center On Jul 10, 2008, at 5:36 PM, Hal Rosenstock wrote: > On Thu, 2008-07-10 at 17:27 -0400, Charles Taylor wrote: >> We've got a couple of Cisco 7008/Topspin 270 core switches. Much >> of >> the diagnostic and topology information we get from the ib diag tools >> in OFED refer to the internal ports rather than the actual external >> slot/port. > > Does ibnetdiscover -g work ? There was some work done to do this > mapping for Cisco but it may be out of date. > > -- Hal > >> We could deduce the mappings, I suppose, by tracing cables and >> using ibtracert between nodes but that seems like a real >> pain. I was wondering if anyone on this list knows how that >> mapping >> works. The chassis itself consists of 8 "ib port" blades in the >> front labelled as slots 1 - 8. There are then six "fabric >> controller" slots in the back of the chassis labelled 9 - 14. >> Slots 11 and 12 are "special" as they hold the "master" and "standby >> master" fabric controllers. >> >> Seems like the FC's in slots 11 and 12 connect the the FCs in slots >> 9,10,13,14 which in turn connect to the external port blades in slots >> 1 - 8. >> >> Again, all of the diagnostic information refers to the internal FC >> ports and mapping those to the external ports (to which we actually >> connect leaf switches) seems non-trivial. I think there was even a >> document somewhere that described the mapping. We just can't find >> it >> now. >> >> Thanks, >> >> Charlie Taylor >> UF HPC Center >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general > _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: TS270_port_mapping.txt URL: From jon at opengridcomputing.com Fri Jul 11 09:39:39 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Fri, 11 Jul 2008 11:39:39 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807091815.25992.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807082255.01460.olaf.kirch@oracle.com> <20080709143440.GA20798@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> Message-ID: <20080711163938.GB31816@opengridcomputing.com> Hey Olaf, I was able to get the tree working after making a small modification to it (see the patch below). Changing to the SQD state in iWARP will cause the connection to be torn down. Simply ignore this modify_qp calls, and iWARP is happy. Thanks, Jon Signed-off-by: Jon Mason diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index 5835a7e..12a6b0a 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -101,7 +101,7 @@ static void rds_ib_connect_complete(struct rds_connection *conn, struct rdma_cm_ * so that we learn quickly when our credit accounting is * buggy. */ - if (ic->i_flowctl) { + if (ic->i_flowctl && !ic->i_iwarp) { /* It seems we have to take a brief detour through SQD state * in order to change the RNR retry count. */ qp_attr.qp_state = IB_QPS_SQD; From swise at opengridcomputing.com Fri Jul 11 10:00:18 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Fri, 11 Jul 2008 12:00:18 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080711163938.GB31816@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807082255.01460.olaf.kirch@oracle.com> <20080709143440.GA20798@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> <20080711163938.GB31816@opengridcomputing.com> Message-ID: <487791A2.8070209@opengridcomputing.com> Jon Mason wrote: > Hey Olaf, > I was able to get the tree working after making a small modification to > it (see the patch below). > > Changing to the SQD state in iWARP will cause the connection to be torn > down. Simply ignore this modify_qp calls, and iWARP is happy. > > Thanks, > Jon > > Signed-off-by: Jon Mason > > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > index 5835a7e..12a6b0a 100644 > --- a/net/rds/ib_cm.c > +++ b/net/rds/ib_cm.c > @@ -101,7 +101,7 @@ static void rds_ib_connect_complete(struct rds_connection *conn, struct rdma_cm_ > * so that we learn quickly when our credit accounting is > * buggy. > */ > - if (ic->i_flowctl) { > + if (ic->i_flowctl && !ic->i_iwarp) { > /* It seems we have to take a brief detour through SQD state > * in order to change the RNR retry count. */ > qp_attr.qp_state = IB_QPS_SQD; > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > Plus all that RNR tweaking is IB-specific anyway. Steve. From olaf.kirch at oracle.com Fri Jul 11 10:47:48 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Fri, 11 Jul 2008 19:47:48 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080711163938.GB31816@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> <20080711163938.GB31816@opengridcomputing.com> Message-ID: <200807111947.48819.olaf.kirch@oracle.com> On Friday 11 July 2008 18:39:39 Jon Mason wrote: > Hey Olaf, > I was able to get the tree working after making a small modification to > it (see the patch below). > > Changing to the SQD state in iWARP will cause the connection to be torn > down. Simply ignore this modify_qp calls, and iWARP is happy. Ah, very good! I'll give the code a little churn on this end to make sure I didn't break any of the IB china. If that doesn't turn up any nasties, I can at least push the current state to Vlad for inclusion in 1.4. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From aj.guillon at gmail.com Fri Jul 11 11:46:24 2008 From: aj.guillon at gmail.com (Adrien Guillon) Date: Fri, 11 Jul 2008 14:46:24 -0400 Subject: ***SPAM*** Re: [ofa-general] Getting Started with RDMA In-Reply-To: References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> <48734962.8000401@voltaire.com> <9870a2060807100932w713290a6wb818adb085d56689@mail.gmail.com> <487651D6.9080301@gmail.com> Message-ID: <9870a2060807111146g30e5de5l2edb7c2c187943dc@mail.gmail.com> Alright, thanks for helping the n00b everyone. How about IB CM? The page says: "Use of the IB CM requires knowledge the IB CM protocols defined in Chapter 12 of the IB Architecture Spec Release 1.2." How difficult is it in reality to use this library? I assume that this would give better performance than RDMA CM? I also presume that I can do RDMA operations with that, since Infiniband provides them in the standard. Thanks, AJ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Fri Jul 11 12:11:09 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 11 Jul 2008 12:11:09 -0700 Subject: [ofa-general] debian build instructions? In-Reply-To: <4875FB8B.4020507@ec.gc.ca> (John Marshall's message of "Thu, 10 Jul 2008 08:07:39 -0400") References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> Message-ID: > There seem to be a lot of people on this list. Is there > no one working with debian that might be able to > give a pointer? I use Debian for pretty much all my development. However I haven't tried to use OFED -- rather, I have just gotten all the support that I use into the main Debian archive. I'm not sure how much is in Etch but Lenny should be pretty good: there are libibverbs, librdmacm, libmthca, libmlx4, libcxgb3, and libipathverbs packages in the main archive, along with Open MPI 1.2.6 built with IB support. And the 2.6.25 kernel in the archive should have all the kernel drivers you need. I'd be curious to know if there is something else you would need to use a stock Debian install. - R. From dotanba at gmail.com Fri Jul 11 13:19:51 2008 From: dotanba at gmail.com (Dotan Barak) Date: Fri, 11 Jul 2008 22:19:51 +0200 Subject: ***SPAM*** Re: [ofa-general] Getting Started with RDMA In-Reply-To: <9870a2060807111146g30e5de5l2edb7c2c187943dc@mail.gmail.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> <48734962.8000401@voltaire.com> <9870a2060807100932w713290a6wb818adb085d56689@mail.gmail.com> <487651D6.9080301@gmail.com> <9870a2060807111146g30e5de5l2edb7c2c187943dc@mail.gmail.com> Message-ID: <4877C067.6070606@gmail.com> Adrien Guillon wrote: > Alright, thanks for helping the n00b everyone. How about IB CM? The > page says: > > "Use of the IB CM requires knowledge the IB CM protocols defined in > Chapter 12 of the IB Architecture Spec Release 1.2." > > How difficult is it in reality to use this library? I assume that > this would give better performance than RDMA CM? I also presume that > I can do RDMA operations with that, since Infiniband provides them in > the standard. > In the past i connected QPs using the IB CM (sorry, but i don't have this code any more ...), It wasn't complicated IF you understand what all of the QP attributes means (i don't know if a beginner has this knowledge). So i would suggest that a beginner will connect QPs using RDMA CM. If you wish, you can even connect the QPs by exchanging the parameter over TCP sockets (it is not what the IB architects meant but hey, it works) Example for this can be seen in the pinpong examples of libibverbs. > Thanks, > > AJ Thanks Dotan From John.Marshall at ec.gc.ca Fri Jul 11 12:47:44 2008 From: John.Marshall at ec.gc.ca (John Marshall) Date: Fri, 11 Jul 2008 19:47:44 +0000 Subject: [ofa-general] debian build instructions? In-Reply-To: References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> Message-ID: <4877B8E0.7000203@ec.gc.ca> Roland Dreier wrote: > > There seem to be a lot of people on this list. Is there > > no one working with debian that might be able to > > give a pointer? > > I use Debian for pretty much all my development. However I haven't > tried to use OFED -- rather, I have just gotten all the support that I > use into the main Debian archive. I'm not sure how much is in Etch but > Lenny should be pretty good: there are libibverbs, librdmacm, libmthca, > libmlx4, libcxgb3, and libipathverbs packages in the main archive, along > with Open MPI 1.2.6 built with IB support. And the 2.6.25 kernel in the > archive should have all the kernel drivers you need. > > I'd be curious to know if there is something else you would need to use > a stock Debian install. > Hi, I think I've missed what is going on. I just checked out the version numbers of the individual packages which make up ofed-1.3 against what is in the main archive (which packages you are putting in) and, lo and behold, they are pretty close. It was the ofed versioning that threw me off so that I thought the archive packages were quite out of date--and that I would have to rebuild them. Am I right? I need the following: opensm - for a switch infiniband-diags - the diag tools mstflint - to update firmware libib*mad - to support the management tools above libibcommon - " Thanks, John From bs at q-leap.de Fri Jul 11 13:06:53 2008 From: bs at q-leap.de (Bernd Schubert) Date: Fri, 11 Jul 2008 22:06:53 +0200 Subject: [ofa-general] debian build instructions? In-Reply-To: <4877B8E0.7000203@ec.gc.ca> References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> <4877B8E0.7000203@ec.gc.ca> Message-ID: <20080711200653.GA28708@lanczos.q-leap.de> On Fri, Jul 11, 2008 at 07:47:44PM +0000, John Marshall wrote: > Roland Dreier wrote: >> > There seem to be a lot of people on this list. Is there >> > no one working with debian that might be able to >> > give a pointer? >> >> I use Debian for pretty much all my development. However I haven't >> tried to use OFED -- rather, I have just gotten all the support that I >> use into the main Debian archive. I'm not sure how much is in Etch but >> Lenny should be pretty good: there are libibverbs, librdmacm, libmthca, >> libmlx4, libcxgb3, and libipathverbs packages in the main archive, along >> with Open MPI 1.2.6 built with IB support. And the 2.6.25 kernel in the >> archive should have all the kernel drivers you need. >> >> I'd be curious to know if there is something else you would need to use >> a stock Debian install. >> > Hi, > > I think I've missed what is going on. I just checked out the version > numbers of the individual packages which make up ofed-1.3 against > what is in the main archive (which packages you are putting in) and, > lo and behold, they are pretty close. It was the ofed versioning that > threw me off so that I thought the archive packages were quite out > of date--and that I would have to rebuild them. > > Am I right? > > I need the following: > opensm - for a switch > infiniband-diags - the diag tools > mstflint - to update firmware > libib*mad - to support the management tools above > libibcommon - " Except mstflint we have already packaged these. Roland, would you sponsor our uploads (we first need to add proper package descriptions, though). Or could we create an alioth group? Cheers, Bernd From rdreier at cisco.com Fri Jul 11 13:09:37 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 11 Jul 2008 13:09:37 -0700 Subject: [ofa-general] debian build instructions? In-Reply-To: <20080711200653.GA28708@lanczos.q-leap.de> (Bernd Schubert's message of "Fri, 11 Jul 2008 22:06:53 +0200") References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> <4877B8E0.7000203@ec.gc.ca> <20080711200653.GA28708@lanczos.q-leap.de> Message-ID: > Except mstflint we have already packaged these. Roland, would you sponsor our > uploads (we first need to add proper package descriptions, though). Or > could we create an alioth group? I am not a Debian developer either... ask on the debian-mentors mailing list to find a sponsor. - R. From rdreier at cisco.com Fri Jul 11 13:10:35 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 11 Jul 2008 13:10:35 -0700 Subject: [ofa-general] debian build instructions? In-Reply-To: <4877B8E0.7000203@ec.gc.ca> (John Marshall's message of "Fri, 11 Jul 2008 19:47:44 +0000") References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> <4877B8E0.7000203@ec.gc.ca> Message-ID: > I think I've missed what is going on. I just checked out the version > numbers of the individual packages which make up ofed-1.3 against > what is in the main archive (which packages you are putting in) and, > lo and behold, they are pretty close. It was the ofed versioning that > threw me off so that I thought the archive packages were quite out > of date--and that I would have to rebuild them. > > Am I right? Yes, that's right -- everything in Debian now is pretty much the latest upstream version. > I need the following: > opensm - for a switch > infiniband-diags - the diag tools > mstflint - to update firmware > libib*mad - to support the management tools above > libibcommon - " mstflint may not be that fun to package. But anyway all of these should be pretty easy to build from git. - R. From bs at q-leap.de Fri Jul 11 13:14:41 2008 From: bs at q-leap.de (Bernd Schubert) Date: Fri, 11 Jul 2008 22:14:41 +0200 Subject: [ofa-general] debian build instructions? In-Reply-To: References: <487281C6.5070001@ec.gc.ca> <4875FB8B.4020507@ec.gc.ca> <4877B8E0.7000203@ec.gc.ca> <20080711200653.GA28708@lanczos.q-leap.de> Message-ID: <20080711201441.GB28708@lanczos.q-leap.de> On Fri, Jul 11, 2008 at 01:09:37PM -0700, Roland Dreier wrote: > > Except mstflint we have already packaged these. Roland, would you sponsor our > > uploads (we first need to add proper package descriptions, though). Or > > could we create an alioth group? > > I am not a Debian developer either... ask on the debian-mentors mailing > list to find a sponsor. Yeah sure, just would have been easier if you would have been a DD ;) Cheers, Bernd From castea97 at thenativity.com Fri Jul 11 13:29:22 2008 From: castea97 at thenativity.com (Kaylin Soto) Date: Sat, 12 Jul 2008 04:29:22 +0800 Subject: [ofa-general] CertifiedProductListFastShipping Message-ID: <01c8e3d7$db7dfd00$9ba66e7d@castea97> Pay less and get more in our health care store! *Offer expires November 30, 2008. As a valued Windows Live Hotmail customer, we hope you find this Windows Vista Ultimate offer valuable. If you would prefer to no longer receive promotional offers about Windows Vista Ultimate please click here. For general information about how to manage your Communication Preferences with Microsoft please click here. If you have questions about Microsoft privacy policies, please read our online Privacy Statement. Opting out of Microsoft e-mail offers will not affect any newsletters you have requested nor restrict important customer communications concerning your Microsoft products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Fri Jul 11 13:31:04 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 11 Jul 2008 13:31:04 -0700 Subject: [ofa-general] [PATCH for bkl-removal] IB/umad: BKL is not needed for ib_umad_open() Message-ID: Remove explicit lock_kernel() calls and document why the code is safe. Signed-off-by: Roland Dreier --- Hi Jon, more BKL removal for your tree. Should be ready for 2.6.27. diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c index 260b3df..d31e5bb 100644 --- a/drivers/infiniband/core/user_mad.c +++ b/drivers/infiniband/core/user_mad.c @@ -47,7 +47,6 @@ #include #include #include -#include #include @@ -778,23 +777,33 @@ static long ib_umad_compat_ioctl(struct file *filp, unsigned int cmd, } #endif +/* + * ib_umad_open() does not need the BKL: + * + * - umad_port[] accesses are protected by port_lock, the + * ib_umad_port structures are properly reference counted, and + * everything else is purely local to the file being created, so + * races against other open calls are not a proble; + * - the ioctl method does not affect any global state outside of the + * file structure being operated on; + * - the port is added to umad_port[] as the last part of module + * initialization so the open method will either immediately run + * -ENXIO, or all required initialization will be done. + */ static int ib_umad_open(struct inode *inode, struct file *filp) { struct ib_umad_port *port; struct ib_umad_file *file; int ret = 0; - lock_kernel(); spin_lock(&port_lock); port = umad_port[iminor(inode) - IB_UMAD_MINOR_BASE]; if (port) kref_get(&port->umad_dev->ref); spin_unlock(&port_lock); - if (!port) { - unlock_kernel(); + if (!port) return -ENXIO; - } mutex_lock(&port->file_mutex); @@ -823,7 +832,6 @@ static int ib_umad_open(struct inode *inode, struct file *filp) out: mutex_unlock(&port->file_mutex); - unlock_kernel(); return ret; } From rdreier at cisco.com Fri Jul 11 13:54:40 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 11 Jul 2008 13:54:40 -0700 Subject: [ofa-general] [PATCH for bkl-removal v2] IB/umad: BKL is not needed for ib_umad_open() In-Reply-To: (Roland Dreier's message of "Fri, 11 Jul 2008 13:31:04 -0700") References: Message-ID: Remove explicit lock_kernel() calls and document why the code is safe. Signed-off-by: Roland Dreier --- Resending with s/proble/problem/ diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c index 260b3df..d31e5bb 100644 --- a/drivers/infiniband/core/user_mad.c +++ b/drivers/infiniband/core/user_mad.c @@ -47,7 +47,6 @@ #include #include #include -#include #include @@ -778,23 +777,33 @@ static long ib_umad_compat_ioctl(struct file *filp, unsigned int cmd, } #endif +/* + * ib_umad_open() does not need the BKL: + * + * - umad_port[] accesses are protected by port_lock, the + * ib_umad_port structures are properly reference counted, and + * everything else is purely local to the file being created, so + * races against other open calls are not a problem; + * - the ioctl method does not affect any global state outside of the + * file structure being operated on; + * - the port is added to umad_port[] as the last part of module + * initialization so the open method will either immediately run + * -ENXIO, or all required initialization will be done. + */ static int ib_umad_open(struct inode *inode, struct file *filp) { struct ib_umad_port *port; struct ib_umad_file *file; int ret = 0; - lock_kernel(); spin_lock(&port_lock); port = umad_port[iminor(inode) - IB_UMAD_MINOR_BASE]; if (port) kref_get(&port->umad_dev->ref); spin_unlock(&port_lock); - if (!port) { - unlock_kernel(); + if (!port) return -ENXIO; - } mutex_lock(&port->file_mutex); @@ -823,7 +832,6 @@ static int ib_umad_open(struct inode *inode, struct file *filp) out: mutex_unlock(&port->file_mutex); - unlock_kernel(); return ret; } From corbet at lwn.net Fri Jul 11 15:42:11 2008 From: corbet at lwn.net (Jonathan Corbet) Date: Fri, 11 Jul 2008 16:42:11 -0600 Subject: [ofa-general] Re: [PATCH for bkl-removal v2] IB/umad: BKL is not needed for ib_umad_open() In-Reply-To: References: Message-ID: <20080711164211.4414a692@bike.lwn.net> On Fri, 11 Jul 2008 13:54:40 -0700 Roland Dreier wrote: > Remove explicit lock_kernel() calls and document why the code is safe. Applied to the bkl-removal tree and pushed - thanks. jon From gvlastly at btshosting.co.uk Fri Jul 11 18:34:52 2008 From: gvlastly at btshosting.co.uk (Alfredo Horn) Date: Fri, 11 Jul 2008 22:34:52 -0300 Subject: [ofa-general] Be strong and assured Message-ID: <001601c8e3a6$55daad50$00151934@arnaldodls> Turn it to a magic stick of pleasure. Helping people omrove their intimate life. Here! engineries fairbairns exposeriez ermentrude escrituras enthoven's explantion exporter's exportfile extranjera esamaritan farmington -------------- next part -------------- An HTML attachment was scrubbed... URL: From pakmail.barrie at sympatico.ca Fri Jul 11 19:36:32 2008 From: pakmail.barrie at sympatico.ca (Mrs. Rose Blair) Date: Fri, 11 Jul 2008 22:36:32 -0400 Subject: [ofa-general] **2008 Winner!!! Message-ID: <20080712023632.MTDU1584.tomts44-srv.bellnexxia.net@toip35-bus.srvr.bell.ca> SWISS ON-LINE LOTTERY P O Box 1010 Liverpool, L70 1NL UNITED KINGDOM REF NO: SL/74/368/05 BATCH NO:SL-121-LT-11-12-05 You won the sum of £750,000.00 GBP from our monthly PROGRAM, held on the 10th July 2008,and released today 11th July 2008.You are adviced to Contact Mr.Michael Anderson to claim your prize. Provide him with your 1,Names: 2,Address: 3,Age: 4,Occupation, 5,Tel: 6,Country: Contact Email: mic_anderson002 at btinternet.com Sincerely, Mrs.Rose Blair.(Swiss Lottery Online Co-ordinator) From vlad at lists.openfabrics.org Sat Jul 12 02:39:59 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sat, 12 Jul 2008 02:39:59 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080712-0200 daily build status Message-ID: <20080712093959.9ACF5E2800E@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2369: error: for each function it appears in.) /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2371: error: 'class_device_attr_mem_info' undeclared (first use in this function) /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.c:2452: error: implicit declaration of function 'class_device_unregister' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt/ib_srpt.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband/ulp/srpt] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080712-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From 4sUKZcarlson at locationet.com Sat Jul 12 08:12:59 2008 From: 4sUKZcarlson at locationet.com (hendrik reza) Date: Sat, 12 Jul 2008 15:12:59 +0000 Subject: [ofa-general] MSG ID:19495 Luxury Footwear (Chanel, Gucci, Armani) for the Priveleged Message-ID: <000601c8e440$07b4d42f$98b536b2@aixlwh> The world's largest luxury store for shoes and bags is just one click away. Recommended by thousands of satisfied customers worldwide, we carry dozens of famous brands including: ~ Louis Vuitton ~ Armani ~ Gucci ~ Prada ~ Hermes Here you will find thousands of stunning designs for shoes, and leather products, at rock bottom pricing. Prices range from just $39 to $199; quality is assured and satisfaction absolutely guaranteed. Sale ends this week, so visit us today and start pampering yourself and your loved ones! - Visit our site: www.shoesgreat[DOT]com (copy this link and then replace "[DOT]" to ".") From ogerlitz at voltaire.com Sat Jul 12 22:25:20 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 13 Jul 2008 08:25:20 +0300 Subject: ***SPAM*** Re: [ofa-general] Getting Started with RDMA In-Reply-To: <9870a2060807111146g30e5de5l2edb7c2c187943dc@mail.gmail.com> References: <9870a2060807061102l458919cq66ec85258ed50233@mail.gmail.com> <4871FFA2.1030700@voltaire.com> <9870a2060807070829k4392980an4f48dc1661dca54@mail.gmail.com> <48734962.8000401@voltaire.com> <9870a2060807100932w713290a6wb818adb085d56689@mail.gmail.com> <487651D6.9080301@gmail.com> <9870a2060807111146g30e5de5l2edb7c2c187943dc@mail.gmail.com> Message-ID: <487991C0.4060002@voltaire.com> Adrien Guillon wrote: > How difficult is it in reality to use this library? I assume that > this would give better performance than RDMA CM? I also presume that > I can do RDMA operations with that, since Infiniband provides them in > the standard. Among other knocks, if you choose to use libibcm then your addressing model should be built upon IB GIDs and you would have to emulate IB L2 route lookup (IB SA queries) since there's no libibsa existing - wheres if you use librdmacm you can use IP addresses in your addressing scheme and can issue a real IB route lookup. Needless to say that your code would work on all supported RDMA transports (currently IB and iWARP). Both libibcm and librdmacm emulate/add nothing in the access to the data path, that is with both you create and use an (say) RC QP created by libibverbs and later work with the low level device specific (eg NetEffect, Mellanox, IBM, Chelsio, Qlogic, etc) user space library (through libibverbs proxy-ing the data calls) - so in that respect no performance difference is introduced by either of them. Or. From tziporet at dev.mellanox.co.il Sun Jul 13 01:53:28 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 13 Jul 2008 11:53:28 +0300 Subject: [ofa-general] tarfiles for libehca, libsdp? In-Reply-To: <200807111149.43036.ossrosch@linux.vnet.ibm.com> References: <20080709165522.9760B1C8E49@ece06.nas.nasa.gov> <200807111149.43036.ossrosch@linux.vnet.ibm.com> Message-ID: <4879C288.4020908@mellanox.co.il> Stefan Roscher wrote: > Hi Bryan, > > > >> >From where can I download the tar files for libehca and libsdp? They are >> not available under the http://www.openfabrics.org/downloads/ directory. >> > > For the libehca please download the http://www.openfabrics.org/downloads/OFED/ofed-1.3/OFED-1.3.tgz file. > In OFED-1.3/SRPMS/ directory you will find the src rpm for libehca. > > Same for libsdp Tziporet From vlad at lists.openfabrics.org Sun Jul 13 02:43:41 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sun, 13 Jul 2008 02:43:41 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080713-0204 daily build status Message-ID: <20080713094341.5A6A1E60383@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080713-0204_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From dotanba at gmail.com Sat Jul 12 23:24:33 2008 From: dotanba at gmail.com (Dotan Barak) Date: Sun, 13 Jul 2008 09:24:33 +0300 Subject: ***SPAM*** Re: [ofa-general] Infinband and DRBD In-Reply-To: References: Message-ID: <2f3bf9a60807122324j60e48a9gddf3b01612c20d0b@mail.gmail.com> I don't know which OFED version is installed in the Distribution version that you are using, but i think (as a general rule, at least at this stage): users should install the latest OFED release if this version number is higher than the one that originally being installed with your distribution. I hope that i answered your answer ... Dotan On Wed, Jul 9, 2008 at 5:12 PM, Robert Dunkley wrote: > Hi Everyone, > > > > Apologies if this question is a bit simple but if anyone has any thoughts > I'd appreciated them. > > > > This is my first Infiniband experiment. I need to clone disk partitions > between Linux boxes, theoretically 400-500Mbytes/second writes could be seen > at peak (Total for all partitions) but loading will mostly be smallish > random writes. I was thinking of just using Centos with the built-in IPOIB > support and DRBD but does anyone have any better suggestions? > > > > Thanks, > > > > Rob > > > > The SAQ Group > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > SEMTEC Limited Trading as SAQ is Registered in England & Wales > Company Number: 06481952 > > > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication and I.T. > services to UK Business. > > DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups > : Managed Networks : Remote Support. > > > > Find us in http://www.thebestof.co.uk/petersfield > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From yevgenyp at mellanox.co.il Sun Jul 13 08:33:56 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Sun, 13 Jul 2008 18:33:56 +0300 Subject: [ofa-general][PATCH 8/11 v4] mlx4: Default value for automatic completion vector selection Message-ID: <487A2064.8070708@mellanox.co.il> mlx4: Default value for automatic completion vector selection When the vector number passed to mlx4_cq_alloc is MLX4_LEAST_ATTACHED_VECTOR (0xffffffff), the driver selects the completion vector that has the least CQ's attached to it and attaches the CQ to the chosen vector. IB_CQ_VECTOR_LEAST_ATTACHED is defined in rdma/ib_verbs.h, when mlx4_ib driver, receives this cq vector number, it uses MLX4_LEAST_ATTACHED_VECTOR at CQ creation. Signed-off-by: Yevgeny Petrilin --- drivers/infiniband/hw/mlx4/cq.c | 4 +++- drivers/net/mlx4/cq.c | 22 +++++++++++++++++++++- drivers/net/mlx4/mlx4.h | 1 + include/linux/mlx4/device.h | 2 ++ include/rdma/ib_verbs.h | 10 +++++++++- 5 files changed, 36 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index dbfa1a3..12945c1 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -221,7 +221,9 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector } err = mlx4_cq_alloc(dev->dev, entries, &cq->buf.mtt, uar, - cq->db.dma, &cq->mcq, vector, 0); + cq->db.dma, &cq->mcq, + vector == IB_CQ_VECTOR_LEAST_ATTACHED ? + MLX4_LEAST_ATTACHED_VECTOR : vector, 0); if (err) goto err_dbmap; diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 18bab87..e0dd18b 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -187,6 +187,22 @@ int mlx4_cq_resize(struct mlx4_dev *dev, struct mlx4_cq *cq, } EXPORT_SYMBOL_GPL(mlx4_cq_resize); +static int mlx4_find_least_loaded_vector(struct mlx4_priv *priv) +{ + int i; + int index = 0; + int min = priv->eq_table.eq[MLX4_EQ_COMP_CPU0].load; + + for (i = 1; i < priv->dev.caps.num_comp_vectors; i++) { + if (priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + i].load < min) { + index = i; + min = priv->eq_table.eq[MLX4_EQ_COMP_CPU0 + i].load; + } + } + + return index; +} + int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, struct mlx4_uar *uar, u64 db_rec, struct mlx4_cq *cq, unsigned vector, int collapsed) @@ -228,7 +244,9 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, cq_context->flags = cpu_to_be32(!!collapsed << 18); cq_context->logsize_usrpage = cpu_to_be32((ilog2(nent) << 24) | uar->index); - if (vector >= dev->caps.num_comp_vectors) { + if (vector == MLX4_LEAST_ATTACHED_VECTOR) + vector = mlx4_find_least_loaded_vector(priv); + else if (vector >= dev->caps.num_comp_vectors) { err = -EINVAL; goto err_radix; } @@ -248,6 +266,7 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, if (err) goto err_radix; + priv->eq_table.eq[cq->comp_eq_idx].load++; cq->cons_index = 0; cq->arm_sn = 1; cq->uar = uar; @@ -285,6 +304,7 @@ void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq) mlx4_warn(dev, "HW2SW_CQ failed (%d) for CQN %06x\n", err, cq->cqn); synchronize_irq(priv->eq_table.eq[cq->comp_eq_idx].irq); + priv->eq_table.eq[cq->comp_eq_idx].load--; spin_lock_irq(&cq_table->lock); radix_tree_delete(&cq_table->tree, cq->cqn); diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 727a48a..c77b5ed 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -144,6 +144,7 @@ struct mlx4_eq { u16 irq; u16 have_irq; int nent; + int load; struct mlx4_buf_list *page_list; struct mlx4_mtt mtt; }; diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 7cbe078..d2ce379 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -151,6 +151,8 @@ enum { MLX4_NUM_FEXCH = 64 * 1024, }; +#define MLX4_LEAST_ATTACHED_VECTOR 0xffffffff + static inline u64 mlx4_fw_ver(u64 major, u64 minor, u64 subminor) { return (major << 32) | (minor << 16) | subminor; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 4c3c76b..b63e2a2 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1393,6 +1393,13 @@ static inline int ib_post_recv(struct ib_qp *qp, return qp->device->post_recv(qp, recv_wr, bad_recv_wr); } +/* + * IB_CQ_VECTOR_LEAST_ATTACHED: The constatnt specifies that + * teh cq will be attached to the least attached + * completion vector + */ +#define IB_CQ_VECTOR_LEAST_ATTACHED 0xffffffff + /** * ib_create_cq - Creates a CQ on the specified device. * @device: The device on which to create the CQ. @@ -1404,7 +1411,8 @@ static inline int ib_post_recv(struct ib_qp *qp, * the associated completion and event handlers. * @cqe: The minimum size of the CQ. * @comp_vector - Completion vector used to signal completion events. - * Must be >= 0 and < context->num_comp_vectors. + * Must be >= 0 and < context->num_comp_vectors + * or IB_CQ_VECTOR_LEAST_ATTACHED. * * Users can examine the cq structure to determine the actual CQ size. */ -- 1.5.3.7 From tequila_14 at hotmail.com Sun Jul 13 09:55:24 2008 From: tequila_14 at hotmail.com (Lene rohwer ) Date: Sun, 13 Jul 2008 19:55:24 +0300 Subject: [ofa-general] Klitzekleiner Zeitaufwand, der Kohle bringt Message-ID: <372464794.18720914411234@hotmail.com> Sie suchen Nebenjob? Wir helfen Ihnen: elfingo at inbox.ru From ronli.voltaire at gmail.com Sun Jul 13 10:13:08 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Sun, 13 Jul 2008 20:13:08 +0300 Subject: [ofa-general] Problem building libmlx4 In-Reply-To: <3b5e77ad0807100521y3e1d0f5w3e97be8dfda64430@mail.gmail.com> References: <3b5e77ad0807100521y3e1d0f5w3e97be8dfda64430@mail.gmail.com> Message-ID: <3b5e77ad0807131013s67ad81ak6bcfcddda092df84@mail.gmail.com> Ok, I just had to set LDFLAGS env variable to: -Xlinker -R/usr/local/lib Now everything works as it should. Ron On Thu, Jul 10, 2008 at 3:21 PM, Ron Livne wrote: > I edited the configure script, canceled the sizeof long test, > and manually entered there that the size is 8. > > everything seems to be working fine now. > > Ron > > > On Thu, Jul 10, 2008 at 2:15 PM, Ron Livne wrote: >> Hi, >> >> I pulled libmlx4 from: >> git://git.kernel.org/pub/scm/libs/infiniband/libmlx4.git >> >> I tried building it with: >> ./autogen.sh >> ./configure >> >> >> but the configure script renders the following error: >> checking size of long... configure: error: cannot compute sizeof (long), >> 77 >> >> I'm using Redhat 5.1 with kernel 2.6.26-rc2 on an x86-64 machine. >> >> Do you have an idea for solving this? >> >> Thank you, >> Ron >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > From rdreier at cisco.com Sun Jul 13 16:37:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 13 Jul 2008 16:37:05 -0700 Subject: [ofa-general] Re: having problems with the latest for-2.6.27 tree In-Reply-To: <4875DCCC.7020404@voltaire.com> (Or Gerlitz's message of "Thu, 10 Jul 2008 12:56:28 +0300") References: <487500A9.5050707@opengridcomputing.com> <4875DCCC.7020404@voltaire.com> Message-ID: > I am not sure to follow what does "class" means in this context, is > this struct rdma_id_private? such that "inside another lock of the > same class" means locking the same mutex for another instance of > this structure? Yes, that is the level that lockdep tracks things on. Technically it is tracking things by "callsite where lock is initialized" I believe. - R. From hr at netultimate.com Sun Jul 13 15:47:21 2008 From: hr at netultimate.com (Sounder Dilipan) Date: Sun, 13 Jul 2008 18:47:21 -0400 Subject: [ofa-general] Work from Home Part time / Full time Remotely working Opportunities Message-ID:

Hi,

We are looking for individuals to work from home in part time remotely for the following positions.

1. Web Designers

2. Web Developers

3. Secretary

4. Tele Sales

5. Data entry

ace="Arial Black" size=4>If you are your friends are If you are your friends are looking for opportunities pl. feel free to reach us at 'yahoo messenger' and id is 'dilipan at yahoo.com ' or email us at hr at netultimate.com

All the best

Regards,

Dilipan

ize=4>If you are your friends are 703 849 1269 (USA)

416 238 0270 (CANADA)

dilipan at netultimate.com

www.netultimate.com


Send Mail To hr at netultimate.com to unsubscribe.
-------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Sun Jul 13 21:55:55 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 13 Jul 2008 21:55:55 -0700 Subject: [ofa-general] Re: [PATCH][resend] IB/mlx4: Use kzalloc when allocating a new mlx4_ib_qp In-Reply-To: <20080710133439.GA20898@mtls03> (Eli Cohen's message of "Thu, 10 Jul 2008 16:34:39 +0300") References: <20080710133439.GA20898@mtls03> Message-ID: Thanks, applied. > text > before: 24565 > after: 24497 > change: 68 btw scripts/bloat-o-meter gives a nice canonical form for this sort of thing (and makes it really easy to get the numbers in the first place): add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-65 (-65) function old new delta create_qp_common 2024 1959 -65 From rdreier at cisco.com Sun Jul 13 22:52:57 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 13 Jul 2008 22:52:57 -0700 Subject: [ofa-general] Insane spam filtering? Message-ID: > Your mail to 'general' with the subject > > ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > The message headers matched a filter rule Anyone know what's going on here? This same email was sent to linux-kernel as well, and reached that list with no problem (http://lkml.org/lkml/2008/7/14/12) and no "***SPAM***" in the subject line, so I assume something is going haywire in the openfabrics.org mail server... - R. From eli at dev.mellanox.co.il Mon Jul 14 00:08:02 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 14 Jul 2008 10:08:02 +0300 Subject: [ofa-general] Re: [PATCH][resend] IB/mlx4: Use kzalloc when allocating a new mlx4_ib_qp In-Reply-To: References: <20080710133439.GA20898@mtls03> Message-ID: <20080714070802.GE18983@mtls03> On Sun, Jul 13, 2008 at 09:55:55PM -0700, Roland Dreier wrote: > > btw scripts/bloat-o-meter gives a nice canonical form for this sort of > thing (and makes it really easy to get the numbers in the first place): > > add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-65 (-65) > function old new delta > create_qp_common 2024 1959 -65 I was wondering how you produce this report but I used the size utility. Thanks for sharing this info. From eli at dev.mellanox.co.il Mon Jul 14 00:22:01 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 14 Jul 2008 10:22:01 +0300 Subject: [ofa-general] [PATCH] IB/core: remove an always true condition Message-ID: <20080714072201.GA8348@mtls03> command is an unsigned variable so it can't be negative. Signed-off-by: Eli Cohen --- drivers/infiniband/core/uverbs_main.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 965c4be..00b7574 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -584,8 +584,7 @@ static ssize_t ib_uverbs_write(struct file *filp, const char __user *buf, if (hdr.in_words * 4 != count) return -EINVAL; - if (hdr.command < 0 || - hdr.command >= ARRAY_SIZE(uverbs_cmd_table) || + if (hdr.command >= ARRAY_SIZE(uverbs_cmd_table) || !uverbs_cmd_table[hdr.command] || !(file->device->ib_dev->uverbs_cmd_mask & (1ull << hdr.command))) return -EINVAL; -- 1.5.6 From jackm at dev.mellanox.co.il Mon Jul 14 00:28:20 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 14 Jul 2008 10:28:20 +0300 Subject: [ofa-general] [PATCH] mlx4: Update/add Melanox Technologies copyright lines to mlx4 driver files (for 2.6.27) Message-ID: <200807141028.20625.jackm@dev.mellanox.co.il> mlx4: Update Mellanox copyright lines. Update existing Mellanox copyright lines to 2008, and add such lines to files where they are missing. Signed-off-by: Jack Morgenstein --- diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c index c75ac94..2906e82 100644 --- a/drivers/infiniband/hw/mlx4/ah.c +++ b/drivers/infiniband/hw/mlx4/ah.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 299f208..4d4a859 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/doorbell.c b/drivers/infiniband/hw/mlx4/doorbell.c index 8aee423..508e464 100644 --- a/drivers/infiniband/hw/mlx4/doorbell.c +++ b/drivers/infiniband/hw/mlx4/doorbell.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c index cdca3a5..f7fc27d 100644 --- a/drivers/infiniband/hw/mlx4/mad.c +++ b/drivers/infiniband/hw/mlx4/mad.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index bcf5064..f766d8d 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index c4cf5b6..71c2a27 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c index 68e9248..5605c56 100644 --- a/drivers/infiniband/hw/mlx4/mr.c +++ b/drivers/infiniband/hw/mlx4/mr.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 89eb6cb..5a83cc3 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c index 12d6bc6..d425652 100644 --- a/drivers/infiniband/hw/mlx4/srq.c +++ b/drivers/infiniband/hw/mlx4/srq.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/user.h b/drivers/infiniband/hw/mlx4/user.h index e2d11be..13beede 100644 --- a/drivers/infiniband/hw/mlx4/user.h +++ b/drivers/infiniband/hw/mlx4/user.h @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/alloc.c b/drivers/net/mlx4/alloc.c index f9d6b4d..096bca5 100644 --- a/drivers/net/mlx4/alloc.c +++ b/drivers/net/mlx4/alloc.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/catas.c b/drivers/net/mlx4/catas.c index aa95287..f094ee0 100644 --- a/drivers/net/mlx4/catas.c +++ b/drivers/net/mlx4/catas.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c index 70dff94..bc40331 100644 --- a/drivers/net/mlx4/cmd.c +++ b/drivers/net/mlx4/cmd.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 95e87a2..9bb50e3 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -2,7 +2,7 @@ * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2004 Voltaire, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c index e141a15..6d61dab 100644 --- a/drivers/net/mlx4/eq.c +++ b/drivers/net/mlx4/eq.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index 2b5006b..8f1b61e 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index a0e046c..c548ead 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c index 2a5bef6..baf4bf6 100644 --- a/drivers/net/mlx4/icm.c +++ b/drivers/net/mlx4/icm.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/icm.h b/drivers/net/mlx4/icm.h index 6c44edf..ab56a2f 100644 --- a/drivers/net/mlx4/icm.h +++ b/drivers/net/mlx4/icm.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/intf.c b/drivers/net/mlx4/intf.c index 4a6c4d5..0e7eb10 100644 --- a/drivers/net/mlx4/intf.c +++ b/drivers/net/mlx4/intf.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index d373601..af2debc 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -1,7 +1,7 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/mcg.c b/drivers/net/mlx4/mcg.c index b4b5787..c83f88c 100644 --- a/drivers/net/mlx4/mcg.c +++ b/drivers/net/mlx4/mcg.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a4023c2..2253d41 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -2,7 +2,7 @@ * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2004 Voltaire, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 03a9abc..f93ff35 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/pd.c b/drivers/net/mlx4/pd.c index 3a93c5f..71db01d 100644 --- a/drivers/net/mlx4/pd.c +++ b/drivers/net/mlx4/pd.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/profile.c b/drivers/net/mlx4/profile.c index 9ca42b2..6ea4b1e 100644 --- a/drivers/net/mlx4/profile.c +++ b/drivers/net/mlx4/profile.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c index ee5484c..c49a860 100644 --- a/drivers/net/mlx4/qp.c +++ b/drivers/net/mlx4/qp.c @@ -1,7 +1,7 @@ /* * Copyright (c) 2004 Topspin Communications. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2004 Voltaire, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/reset.c b/drivers/net/mlx4/reset.c index e199715..3951b88 100644 --- a/drivers/net/mlx4/reset.c +++ b/drivers/net/mlx4/reset.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/srq.c b/drivers/net/mlx4/srq.c index d23f46d..533eb6d 100644 --- a/drivers/net/mlx4/srq.c +++ b/drivers/net/mlx4/srq.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU From olaf.kirch at oracle.com Mon Jul 14 01:01:29 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Mon, 14 Jul 2008 10:01:29 +0200 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <20080711163938.GB31816@opengridcomputing.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> <20080711163938.GB31816@opengridcomputing.com> Message-ID: <200807141001.30900.olaf.kirch@oracle.com> On Friday 11 July 2008 18:39:39 Jon Mason wrote: > Changing to the SQD state in iWARP will cause the connection to be torn > down. Simply ignore this modify_qp calls, and iWARP is happy. I guess actually both branches of the if statement could be skipped for iWARP - there's no point in tweaking the RNR counter for iWARP, even if it doesn't hurt on cxgb3, right? Olaf > > Thanks, > Jon > > Signed-off-by: Jon Mason > > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > index 5835a7e..12a6b0a 100644 > --- a/net/rds/ib_cm.c > +++ b/net/rds/ib_cm.c > @@ -101,7 +101,7 @@ static void rds_ib_connect_complete(struct rds_connection *conn, struct rdma_cm_ > * so that we learn quickly when our credit accounting is > * buggy. > */ > - if (ic->i_flowctl) { > + if (ic->i_flowctl && !ic->i_iwarp) { > /* It seems we have to take a brief detour through SQD state > * in order to change the RNR retry count. */ > qp_attr.qp_state = IB_QPS_SQD; > -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From 32st.pilon at ville.laval.qc.ca Mon Jul 14 18:33:42 2008 From: 32st.pilon at ville.laval.qc.ca (eduino hampton) Date: Tue, 15 Jul 2008 01:33:42 +0000 Subject: [ofa-general] to general Message-ID: <000801c8e629$04b8da8c$4e0ddeb0@iscysulc> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.rahwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From amirv at mellanox.co.il Mon Jul 14 02:15:40 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Mon, 14 Jul 2008 12:15:40 +0300 Subject: [ofa-general] cma: add support for TIMEWAIT_EXIT event (second try) In-Reply-To: <001201c8dc74$2b8c6f40$b2fc070a@amr.corp.intel.com> References: <48590FF2.8090507@mellanox.co.il> <001201c8dc74$2b8c6f40$b2fc070a@amr.corp.intel.com> Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD14E87F@mtlexch01.mtl.com> Sean Hi, Acquiring the handler_mutex explicitly sounds like a good idea. When will it be commited? Will we have it in 2.6.27? Thanks, - Amir > -----Original Message----- > From: Sean Hefty [mailto:sean.hefty at intel.com] > Sent: Wednesday, July 02, 2008 9:48 PM > To: Amir Vadai > Cc: general at lists.openfabrics.org; Oren Duer > Subject: RE: [ofa-general] cma: add support for TIMEWAIT_EXIT > event (second try) > > >SDP needs to be notified when CM exits the TIMEWAIT state. > >CM does notify the CMA but CMA doesn't pass it to the upper layer. > > > >Below is a patch for the CMA code. I wasn't sure if it is ok to set > >CMA_DISCONNECT in the IB_CM_TIMEWAIT_EXIT instead of doing it in the > >IB_CM_DREQ_RECEIVED/IB_CM_DREP_RECEIVED as it is done today. Instead > >I changed the check at the beginning of cma_ib_handler() as you can > >see in the patch. > > > >Signed-off-by: Amir Vadai > > Sorry I was slow to get to this. The general approach looks > fine to me. > > >--- include/rdma/rdma_cm.h.orig 2008-06-18 > 15:04:22.560904000 +0300 > >+++ include/rdma/rdma_cm.h 2008-06-11 11:55:26.758053000 +0300 > >@@ -53,7 +53,8 @@ enum rdma_cm_event_type { > > RDMA_CM_EVENT_DISCONNECTED, > > RDMA_CM_EVENT_DEVICE_REMOVAL, > > RDMA_CM_EVENT_MULTICAST_JOIN, > >- RDMA_CM_EVENT_MULTICAST_ERROR > >+ RDMA_CM_EVENT_MULTICAST_ERROR, > >+ RDMA_CM_EVENT_TIMWAIT_EXIT > > }; > > This will need to be updated based on the address change > event patch that went > in. I'll update the librdmacm myself separately. > > > enum rdma_port_space { > >--- drivers/infiniband/core/cma.c.orig 2008-06-11 > 11:24:38.021543000 > >+0300 > >+++ drivers/infiniband/core/cma.c 2008-06-18 > 13:27:08.098747000 +0300 > >@@ -931,7 +931,8 @@ static int cma_ib_handler(struct ib_cm_i > > struct rdma_cm_event event; > > int ret = 0; > > > >- if (cma_disable_remove(id_priv, CMA_CONNECT)) > >+ if ((ib_event->event != IB_CM_TIMEWAIT_EXIT && > >cma_disable_remove(id_priv, CMA_CONNECT)) || > >+ (ib_event->event == IB_CM_TIMEWAIT_EXIT && > >cma_disable_remove(id_priv, CMA_DISCONNECT))) > > return 0; > > I wonder if it would be better to acquire the handler_mutex > explicitly, like > this: > > mutex_lock(&id_priv->handler_mutex); > if ((id_priv->state == CMA_CONNECT && > ib_event->event != IB_CM_TIMEWAIT_EXIT) || > (id_priv->state == CMA_DISCONNECT && > ib_event->event == IB_CM_TIMEWAIT_EXIT)) { > mutex_unlock(&id_priv->handler_mutex); > return 0; > } > > so that ensuring the mutex is acquired once isn't so subtle. > > - Sean > > From vlad at lists.openfabrics.org Mon Jul 14 02:41:07 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Mon, 14 Jul 2008 02:41:07 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080714-0200 daily build status Message-ID: <20080714094107.CF637E608BD@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080714-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From jsquyres at cisco.com Mon Jul 14 04:03:04 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Mon, 14 Jul 2008 07:03:04 -0400 Subject: [ofa-general] Insane spam filtering? In-Reply-To: References: Message-ID: Someone floated an idea a while ago of automatically whitelisting every member of the list. Is that not possible? On Jul 14, 2008, at 1:52 AM, Roland Dreier wrote: >> Your mail to 'general' with the subject >> >> ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 >> >> Is being held until the list moderator can review it for approval. >> >> The reason it is being held: >> >> The message headers matched a filter rule > > Anyone know what's going on here? This same email was sent to > linux-kernel as well, and reached that list with no problem > (http://lkml.org/lkml/2008/7/14/12) and no "***SPAM***" in the subject > line, so I assume something is going haywire in the openfabrics.org > mail > server... > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Jeff Squyres Cisco Systems From eli at dev.mellanox.co.il Mon Jul 14 04:11:00 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 14 Jul 2008 14:11:00 +0300 Subject: [ofa-general] IPoIB arps disappearing In-Reply-To: References: Message-ID: <20080714111100.GG18983@mtls03> On Thu, Jul 10, 2008 at 04:30:11AM -0400, Michael Di Domenico wrote: > I'm having a bit of a weird problem that i cannot figure out. If anyone can > help from the community it would be appreciated. > Here's the packet flow > > cn(ib0)->io(ib0)->io(eth5)->pan(*) > > cn = compute node > io = io node > pan = panasas storage network > > We have 12 shelves of panasas network storage on a seperate network, which > is being fronted by bridge servers which are routing IPoIB traffic to 10G > ethernet traffic. We're using Mellanox Connect-X Ethernet/IB adapters > everwhere. We're running Ofed 1.3.1 and the latest firmwares for IB/Eth > everywhere. > > Here's the problem. I can mount the storage on the compute nodes, but if i > try to send anything more then 50MB of data via dd. I seem to loose the ARP > entries for the compute nodes on the IO servers. This seems to happen > whether I use the filesystem or a netperf run from the compute node to the > panasas storage > > I can run netperf between the compute node and io node and get full IPoIB > line rate with no issues > I can run netperf between the io node and the panasas storage and get full > 10G ethernet line rate with no issues > > When looking at the TCP traces, i can clearly see that a big chunk of data > is sent between the end-points and then it stalls. Immediately after the > stall is an ARP request and then another chunk of data, and this scenario > repeats over and over. > > Any thoughts or questions? > For the benefit of all - the issue was resolved by loading the mtnic module with lro=0. It should be a general guidance to disable LRO for hosts that are configured to route packets. The reason for this is that LRO aggregates IP packets related to the same TCP stream to a single IP packet comprised of a list SKBs, and by doing that, it does not recalculate the TCP checksum. Since routing is done at the IP layer, the checksum on outgoing packets is not recalculated and the final destination will receive packets with bas TCP checksum. From ogerlitz at voltaire.com Mon Jul 14 04:29:49 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 14 Jul 2008 14:29:49 +0300 Subject: [ofa-general] IPoIB arps disappearing - LRO / IP forwarding In-Reply-To: <20080714111100.GG18983@mtls03> References: <20080714111100.GG18983@mtls03> Message-ID: <487B38AD.9020809@voltaire.com> Eli Cohen wrote: > For the benefit of all - the issue was resolved by loading the mtnic module with lro=0. It should be a general guidance to disable LRO for hosts that are configured to route packets. The reason for this is that LRO aggregates IP packets related to the same TCP stream to a single IP packet comprised of a list SKBs, and by doing that, it does not recalculate the TCP checksum. Since routing is done at the IP layer, the checksum on outgoing packets is not recalculated and the > final destination will receive packets with bas TCP checksum. So with the recent patch to ipoib which adds LRO, should we make the default being LRO off? or maybe the approach brought by this patch "net: Disable LRO on devices that are forwarding" http://marc.info/?l=linux-netdev&m=121389887214425&w=2 would solve the problem without the user intervention? Or. From vlad at mellanox.co.il Mon Jul 14 05:37:27 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Mon, 14 Jul 2008 15:37:27 +0300 Subject: [ofa-general] [Fwd: [PATCH] Add nfsrdma_mod to config/build options] In-Reply-To: <487693EF.8030005@nasa.gov> References: <487693EF.8030005@nasa.gov> Message-ID: <487B4887.9010309@mellanox.co.il> Jeff Becker wrote: > Hi all. I tried git-send-email to send this, and I thought it would give > me a chance to edit the patch. This is the first patch to integrate > NFSRDMA into OFED 1.4. I tested that it builds correctly with 2.6.26-rc6 > (current OFED 1.4 base). Now onto the backports! > > -jeff > > Add nfsrdma_mod to config/build options > > Signed-off-by: Jeff Becker > > --- > ofed_scripts/Makefile | 7 ++ > ofed_scripts/configure | 170 > +++++++++++++++++++++++++++++++++++++++++ > ofed_scripts/makefile | 66 ++++++++++++++++ > ofed_scripts/ofed_checkout.sh | 24 ++++++ > 4 files changed, 267 insertions(+), 0 deletions(-) > Applied to ofed_1_4/linux-2.6.git Regards, Vladimir From cap at nsc.liu.se Mon Jul 14 06:15:07 2008 From: cap at nsc.liu.se (Peter Kjellstrom) Date: Mon, 14 Jul 2008 15:15:07 +0200 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080709204617.GR12551@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> Message-ID: <200807141515.11717.cap@nsc.liu.se> On Wednesday 09 July 2008, Sasha Khapyorsky wrote: > On 13:22 Wed 09 Jul , Hal Rosenstock wrote: ... > > Is it > > just the *.ver file or also the version in the *.map file ? > > It is just *.ver file, *.map is symbol versioning - there were additions, > no changes in existing functions API. > > > > this will take effect in the next daily build. > > > > The tar balls just issued won't be reissued ? > > There were some commits after this already. Am I understanding this correctly, you posted tarball.x.y.z.tar.gz and at a later time modified it but left the old name? If so please don't. It's very nasty to handle situations where there are a bad _and_ a fixed version of the same file. It's not like you will run out of numbers ju bump it to x.y.z+1 (or possibly x.y.z.1). Sorry for the noise if I misunderstood. /Peter > Sasha -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rdreier at cisco.com Sun Jul 13 22:16:08 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 13 Jul 2008 22:16:08 -0700 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 Message-ID: I've been a little lazy about writing this, and 2.6.26 is already out, so it's time to review what my plans are for the merge window opens. I have to say that I've been somewhat disappointed with the quality of things this cycle. Maybe I've just been in a tetchy mood these days, but before you send me an email saying, "this is ready to go, when are you going to merge it," please try doing some of the following; - Run your patch through checkpatch.pl so I don't have to nag you to fix trivial issues (or spend time fixing them myself). - Read your patch over so I don't see a memory leak or deadlock as soon as I look at it. - Build your patch with sparse checking ("C=2 CF=-D__CHECK_ENDIAN__") and make sure it doesn't introduce new warnings. (A big bonus in goodwill for sending patches that fix old warnings) - Test your patch on a kernel with things like slab debugging and lockdep turned on. And while you're waiting for me to get to your patch, I sure wouldn't mind if you read and commented on someone else's patch. None of this means you shouldn't remind me about pending patches, since I often lose track of things and drop them accidentally. Anyway, here are all the pending things that I'm aware of. As usual, if something isn't already in my tree and isn't listed below, I probably missed it or dropped it by mistake. Please remind me again in that case. Core: - I'm waiting to merge the RDMA_CM_EVENT_ADDR_CHANGE changes that depend on core networking changes until such changes are upstream. Or, please remind me when that happens. - Jack's XRC patch set. I think we're getting closer to converging here, and I hope to get this merged but we're getting down to the wire, so we'll see. ULPs: - I merged a decent amount of stuff for IPoIB, including LRO, "loopback blocking," and better handling of fabric events, and I don't think I have anything pending. HW specific: - Yevgeny's mlx4 changes. We'll see how much time is left after I get done with XRC (which is before this on my list) but to be honest I'm not sure how mergeable a lot of this is without the mlx4_en patches that actually use it. - I've been working on memory management extensions support for mlx4, but I'm not sure if it will be ready in time. Firmware for this may not be released for a while so it ain't urgent anyway. Here are a few topics that I believe will not be ready in time for the 2.6.27 window and will need to wait for 2.6.28 at least: - Remove LLTX from IPoIB. I haven't had time to finish this yet, so I guess it will probably wait for 2.6.28 now... - Multiple CQ event vector support. No one has convinced me that we know how ULPs or userspace apps should decide which vector to use, and hence little progress has been made since we deferred this during the 2.6.23 merge window. Here all the patches I already have in my for-2.6.27 branch: Christophe Jaillet (1): RDMA/nes: Remove unnecessary memset() Dotan Barak (1): RDMA: Improve include file coding style Eli Cohen (12): IB/mlx4: Optimize QP stamping IPoIB: Copy small received SKBs in connected mode IB/mlx4: Configure QPs' max message size based on real device capability IB/mlx4: Pass congestion management class MADs to the HCA IPoIB: Remove unused IPOIB_MCAST_STARTED code IPoIB: Remove priv->mcast_mutex IPoIB: Only set Q_Key once: after joining broadcast group IPoIB: Use rtnl lock/unlock when changing device flags IPoIB: Use dev_set_mtu() to change mtu IPoIB/cm: Reduce connected mode TX object size IPoIB: Double default RX/TX ring sizes IB/mlx4: Use kzalloc() for new QPs so flags are initialized to 0 Joachim Fenkes (2): IB/ehca: Reject receive work requests if QP is in RESET state IB/ehca: Make device table externally visible Jon Mason (1): RDMA/cxgb3: Propagate HW page size capabilities Moni Shoua (2): IB/sa: Fail requests made while creating new SM AH IPoIB: Refresh paths instead of flushing them on SM change events Or Gerlitz (2): RDMA/addr: Keep pointer to netdevice in struct rdma_dev_addr RDMA/cma: Simplify locking needed for serialization of callbacks Ralph Campbell (2): IB/core: Reset to error QP state transition is not allowed IB/ipath: Use IEEE OUI for vendor_id reported by ibv_query_device() Robert P. J. Day (1): IB/ipath: Simplify code using ARRAY_SIZE() macro Roland Dreier (13): IB/srp: Remove use of cached P_Key/GID queries RDMA: Remove subversion $Id tags IB/mthca: Remove extra code for RESET->ERR QP state transition IB/mlx4: Remove extra code for RESET->ERR QP state transition RDMA/cxgb3: Remove write-only iwch_rnic_attributes fields RDMA/cma: Add missing newlines to printk()s IPoIB/cm: Fix racy use of receive WR/SGL in ipoib_cm_post_receive_nonsrq() RDMA/nes: Encapsulate logic nes_put_cqp_request() RDMA/nes: Get rid of ring_doorbell parameter of nes_post_cqp_request() IPoIB: Get rid of ipoib_mcast_detach() wrapper IB/mthca: Remove "stop" flag for catastrophic error polling timer IB/mthca: Use round_jiffies() for catastrophic error polling timer IB/mthca: Fix check of max_send_sge for special QPs Ron Livne (3): IB/core: Add support for multicast loopback blocking IB/mlx4: Add support for blocking multicast loopback packets IPoIB: Use multicast loopback blocking if available Sean Hefty (1): RDMA: Fix license text Stefan Roscher (1): IB/ehca: In case of lost interrupts, trigger EOI to reenable interrupts Steve Wise (8): RDMA/core: Add memory management extensions support RDMA/cxgb3: MEM_MGT_EXTENSIONS support RDMA/cxgb3: Fix up some ib_device_attr fields RDMA/core: Add iWARP protocol statistics attributes in sysfs RDMA/cxgb3: Add support for protocol statistics RDMA/cxgb3: Set rkey field for new memory windows in iwch_alloc_mw() RDMA/core: Add local DMA L_Key support RDMA/cxgb3: Fixes for zero STag Vladimir Sokolovsky (2): IPoIB: add LRO support mlx4_core: Use MOD_STAT_CFG command to get minimal page size From ronli.voltaire at gmail.com Mon Jul 14 00:59:09 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Mon, 14 Jul 2008 10:59:09 +0300 Subject: [ofa-general] ***SPAM*** Re: [PATCH 3/12 V2] libmlx4: XRC base implementation In-Reply-To: <200807101851.30229.jackm@dev.mellanox.co.il> References: <200807101851.30229.jackm@dev.mellanox.co.il> Message-ID: <3b5e77ad0807140059h52fb3ac6g184307b54bf70290@mail.gmail.com> Hi Jack: +#ifdef HAVE_IBV_XRC_OPS + context->ibv_ctx.more_ops = &mlx4_more_ops; +#endif As you know, I'm using more_ops for my new ibv_create_qp_expanded user verb. I need the context->ibv_ctx.more_ops to be always initialized, even if HAVE_IBV_XRC_OPS isn't set. I'll release a patch for it. Ron On Thu, Jul 10, 2008 at 6:51 PM, Jack Morgenstein wrote: > From 33c27150ed20ff34622a45abd5ff27727f4c34aa Mon Sep 17 00:00:00 2001 > From: Jack Morgenstein > Date: Wed, 9 Jul 2008 13:55:24 +0300 > Subject: [PATCH] XRC implementation, consolidated: > > xrc ops were moved to their own structure at the end of > struct ibv_context (to preserve binary compatibility). > > Check for ibv_context.xrc_ops member via AC_CHECK_MEMBER > > XRC QPs have MSB set in qp number, for identification in > completion handling. > > jackm at mtldesk006:/local/roland/libmlx4> head -n 20 patches/xrc_consolidated.patch > XRC implementation, consolidated: > > xrc ops were moved to their own structure at the end of > struct ibv_context (to preserve binary compatibility). > > Check for ibv_context.xrc_ops member via AC_CHECK_MEMBER > > XRC QPs have MSB set in qp number, for identification in > completion handling. > > Various bug fixes. > (OFED 1.3 commit 39fe7f47e8fc07f356098df048d00740ba585fc5) > > V2: > 1. checkpatch.pl cleanup > 2. Changed xrc_ops to more ops > 3. Check for xrc verbs in ibv_more_ops via AC_CHECK_MEMBER > > Signed-off-by: Jack Morgenstein > --- > configure.in | 3 + > src/cq.c | 35 ++++++++++++-- > src/mlx4-abi.h | 16 ++++++ > src/mlx4.c | 20 ++++++++ > src/mlx4.h | 46 ++++++++++++++++++ > src/qp.c | 7 ++- > src/srq.c | 50 +++++++++++++++++++ > src/verbs.c | 144 +++++++++++++++++++++++++++++++++++++++++++++++++++----- > src/wqe.h | 2 +- > 9 files changed, 303 insertions(+), 20 deletions(-) > > diff --git a/configure.in b/configure.in > index 25f27f7..9304539 100644 > --- a/configure.in > +++ b/configure.in > @@ -42,6 +42,12 @@ AC_CHECK_HEADER(valgrind/memcheck.h, > dnl Checks for typedefs, structures, and compiler characteristics. > AC_C_CONST > AC_CHECK_SIZEOF(long) > +AC_CHECK_MEMBER(struct ibv_context.more_ops, > + [AC_DEFINE([HAVE_IBV_MORE_OPS], 1, [Define to 1 if more_ops is a member of ibv_context])],, > + [#include ]) > +AC_CHECK_MEMBER(struct ibv_more_ops.create_xrc_srq, > + [AC_DEFINE([HAVE_IBV_XRC_OPS], 1, [Define to 1 if have xrc ops])],, > + [#include ]) > > dnl Checks for library functions > AC_CHECK_FUNC(ibv_read_sysfs_file, [], > diff --git a/src/cq.c b/src/cq.c > index 68e16e9..c598b87 100644 > --- a/src/cq.c > +++ b/src/cq.c > @@ -196,10 +196,12 @@ static int mlx4_poll_one(struct mlx4_cq *cq, > struct mlx4_cqe *cqe; > struct mlx4_srq *srq; > uint32_t qpn; > + uint32_t srqn; > uint32_t g_mlpath_rqpn; > uint16_t wqe_index; > int is_error; > int is_send; > + int is_src_recv = 0; > > cqe = next_cqe_sw(cq); > if (!cqe) > @@ -221,20 +223,30 @@ static int mlx4_poll_one(struct mlx4_cq *cq, > is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == > MLX4_CQE_OPCODE_ERROR; > > - if (!*cur_qp || > - (ntohl(cqe->my_qpn) & 0xffffff) != (*cur_qp)->ibv_qp.qp_num) { > + if (qpn & MLX4_XRC_QPN_BIT && !is_send) { > + srqn = ntohl(cqe->g_mlpath_rqpn) & 0xffffff; > + /* > + * We do not have to take the XRC SRQ table lock here, > + * because CQs will be locked while XRC SRQs are removed > + * from the table. > + */ > + srq = mlx4_find_xrc_srq(to_mctx(cq->ibv_cq.context), srqn); > + if (!srq) > + return CQ_POLL_ERR; > + is_src_recv = 1; > + } else if (!*cur_qp || (qpn & 0xffffff) != (*cur_qp)->ibv_qp.qp_num) { > /* > * We do not have to take the QP table lock here, > * because CQs will be locked while QPs are removed > * from the table. > */ > *cur_qp = mlx4_find_qp(to_mctx(cq->ibv_cq.context), > - ntohl(cqe->my_qpn) & 0xffffff); > + qpn & 0xffffff); > if (!*cur_qp) > return CQ_POLL_ERR; > } > > - wc->qp_num = (*cur_qp)->ibv_qp.qp_num; > + wc->qp_num = qpn & 0xffffff; > > if (is_send) { > wq = &(*cur_qp)->sq; > @@ -242,6 +254,10 @@ static int mlx4_poll_one(struct mlx4_cq *cq, > wq->tail += (uint16_t) (wqe_index - (uint16_t) wq->tail); > wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)]; > ++wq->tail; > + } else if (is_src_recv) { > + wqe_index = htons(cqe->wqe_index); > + wc->wr_id = srq->wrid[wqe_index]; > + mlx4_free_srq_wqe(srq, wqe_index); > } else if ((*cur_qp)->ibv_qp.srq) { > srq = to_msrq((*cur_qp)->ibv_qp.srq); > wqe_index = htons(cqe->wqe_index); > @@ -387,6 +403,10 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) > uint32_t prod_index; > uint8_t owner_bit; > int nfreed = 0; > + int is_xrc_srq = 0; > + > + if (srq && srq->ibv_srq.xrc_cq) > + is_xrc_srq = 1; > > /* > * First we need to find the current producer index, so we > @@ -405,7 +425,12 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) > */ > while ((int) --prod_index - (int) cq->cons_index >= 0) { > cqe = get_cqe(cq, prod_index & cq->ibv_cq.cqe); > - if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { > + if (is_xrc_srq && > + (ntohl(cqe->g_mlpath_rqpn & 0xffffff) == srq->srqn) && > + !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) { > + mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); > + ++nfreed; > + } else if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { > if (srq && !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) > mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); > ++nfreed; > diff --git a/src/mlx4-abi.h b/src/mlx4-abi.h > index 20a40c9..1b1253c 100644 > --- a/src/mlx4-abi.h > +++ b/src/mlx4-abi.h > @@ -68,6 +68,14 @@ struct mlx4_resize_cq { > __u64 buf_addr; > }; > > +#ifdef HAVE_IBV_XRC_OPS > +struct mlx4_create_xrc_srq { > + struct ibv_create_xrc_srq ibv_cmd; > + __u64 buf_addr; > + __u64 db_addr; > +}; > +#endif > + > struct mlx4_create_srq { > struct ibv_create_srq ibv_cmd; > __u64 buf_addr; > @@ -90,4 +98,12 @@ struct mlx4_create_qp { > __u8 reserved[5]; > }; > > +#ifdef HAVE_IBV_XRC_OPS > +struct mlx4_open_xrc_domain_resp { > + struct ibv_open_xrc_domain_resp ibv_resp; > + __u32 xrcdn; > + __u32 reserved; > +}; > +#endif > + > #endif /* MLX4_ABI_H */ > diff --git a/src/mlx4.c b/src/mlx4.c > index 34ece39..8a46543 100644 > --- a/src/mlx4.c > +++ b/src/mlx4.c > @@ -68,6 +68,16 @@ struct { > HCA(MELLANOX, 0x673c), /* MT25408 "Hermon" QDR PCIe gen2 */ > }; > > +#ifdef HAVE_IBV_MORE_OPS > +static struct ibv_more_ops mlx4_more_ops = { > +#ifdef HAVE_IBV_XRC_OPS > + .create_xrc_srq = mlx4_create_xrc_srq, > + .open_xrc_domain = mlx4_open_xrc_domain, > + .close_xrc_domain = mlx4_close_xrc_domain, > +#endif > +}; > +#endif > + > static struct ibv_context_ops mlx4_ctx_ops = { > .query_device = mlx4_query_device, > .query_port = mlx4_query_port, > @@ -123,6 +131,15 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ > for (i = 0; i < MLX4_QP_TABLE_SIZE; ++i) > context->qp_table[i].refcnt = 0; > > + context->num_xrc_srqs = resp.qp_tab_size; > + context->xrc_srq_table_shift = ffs(context->num_xrc_srqs) - 1 > + - MLX4_XRC_SRQ_TABLE_BITS; > + context->xrc_srq_table_mask = (1 << context->xrc_srq_table_shift) - 1; > + > + pthread_mutex_init(&context->xrc_srq_table_mutex, NULL); > + for (i = 0; i < MLX4_XRC_SRQ_TABLE_SIZE; ++i) > + context->xrc_srq_table[i].refcnt = 0; > + > for (i = 0; i < MLX4_NUM_DB_TYPE; ++i) > context->db_list[i] = NULL; > > @@ -155,6 +172,9 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ > pthread_spin_init(&context->uar_lock, PTHREAD_PROCESS_PRIVATE); > > context->ibv_ctx.ops = mlx4_ctx_ops; > +#ifdef HAVE_IBV_XRC_OPS > + context->ibv_ctx.more_ops = &mlx4_more_ops; > +#endif > > return &context->ibv_ctx; > > diff --git a/src/mlx4.h b/src/mlx4.h > index 827a201..28a8f9c 100644 > --- a/src/mlx4.h > +++ b/src/mlx4.h > @@ -79,6 +79,11 @@ > > #endif > > +#ifndef HAVE_IBV_MORE_OPS > +#undef HAVE_IBV_XRC_OPS > +#undef HAVE_IBV_CREATE_QP_EXP > +#endif > + > #define HIDDEN __attribute__((visibility ("hidden"))) > > #define PFX "mlx4: " > @@ -97,6 +97,16 @@ enum { > MLX4_QP_TABLE_MASK = MLX4_QP_TABLE_SIZE - 1 > }; > > +enum { > + MLX4_XRC_SRQ_TABLE_BITS = 8, > + MLX4_XRC_SRQ_TABLE_SIZE = 1 << MLX4_XRC_SRQ_TABLE_BITS, > + MLX4_XRC_SRQ_TABLE_MASK = MLX4_XRC_SRQ_TABLE_SIZE - 1 > +}; > + > +enum { > + MLX4_XRC_QPN_BIT = (1 << 23) > +}; > + > enum mlx4_db_type { > MLX4_DB_TYPE_CQ, > MLX4_DB_TYPE_RQ, > @@ -157,6 +167,15 @@ struct mlx4_context { > int qp_table_shift; > int qp_table_mask; > > + struct { > + struct mlx4_srq **table; > + int refcnt; > + } xrc_srq_table[MLX4_XRC_SRQ_TABLE_SIZE]; > + pthread_mutex_t xrc_srq_table_mutex; > + int num_xrc_srqs; > + int xrc_srq_table_shift; > + int xrc_srq_table_mask; > + > struct mlx4_db_page *db_list[MLX4_NUM_DB_TYPE]; > pthread_mutex_t db_list_mutex; > }; > @@ -243,6 +262,11 @@ struct mlx4_ah { > struct mlx4_av av; > }; > > +struct mlx4_xrc_domain { > + struct ibv_xrc_domain ibv_xrcd; > + uint32_t xrcdn; > +}; > + > static inline unsigned long align(unsigned long val, unsigned long align) > { > return (val + align - 1) & ~(align - 1); > @@ -287,6 +311,13 @@ static inline struct mlx4_ah *to_mah(struct ibv_ah *ibah) > return to_mxxx(ah, ah); > } > > +#ifdef HAVE_IBV_XRC_OPS > +static inline struct mlx4_xrc_domain *to_mxrcd(struct ibv_xrc_domain *ibxrcd) > +{ > + return to_mxxx(xrcd, xrc_domain); > +} > +#endif > + > int mlx4_alloc_buf(struct mlx4_buf *buf, size_t size, int page_size); > void mlx4_free_buf(struct mlx4_buf *buf); > > @@ -333,6 +364,10 @@ void mlx4_free_srq_wqe(struct mlx4_srq *srq, int ind); > int mlx4_post_srq_recv(struct ibv_srq *ibsrq, > struct ibv_recv_wr *wr, > struct ibv_recv_wr **bad_wr); > +struct mlx4_srq *mlx4_find_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn); > +int mlx4_store_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn, > + struct mlx4_srq *srq); > +void mlx4_clear_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn); > > struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr); > int mlx4_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, > @@ -363,5 +398,16 @@ int mlx4_alloc_av(struct mlx4_pd *pd, struct ibv_ah_attr *attr, > void mlx4_free_av(struct mlx4_ah *ah); > int mlx4_attach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); > int mlx4_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); > +#ifdef HAVE_IBV_XRC_OPS > +struct ibv_srq *mlx4_create_xrc_srq(struct ibv_pd *pd, > + struct ibv_xrc_domain *xrc_domain, > + struct ibv_cq *xrc_cq, > + struct ibv_srq_init_attr *attr); > +struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, > + int fd, int oflag); > + > +int mlx4_close_xrc_domain(struct ibv_xrc_domain *d); > +#endif > + > > #endif /* MLX4_H */ > diff --git a/src/qp.c b/src/qp.c > index bb98c09..02c5d34 100644 > --- a/src/qp.c > +++ b/src/qp.c > @@ -226,7 +226,7 @@ int mlx4_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, > ctrl = wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1)); > qp->sq.wrid[ind & (qp->sq.wqe_cnt - 1)] = wr->wr_id; > > - ctrl->srcrb_flags = > + ctrl->xrcrb_flags = > (wr->send_flags & IBV_SEND_SIGNALED ? > htonl(MLX4_WQE_CTRL_CQ_UPDATE) : 0) | > (wr->send_flags & IBV_SEND_SOLICITED ? > @@ -243,6 +243,9 @@ int mlx4_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, > size = sizeof *ctrl / 16; > > switch (ibqp->qp_type) { > + case IBV_QPT_XRC: > + ctrl->xrcrb_flags |= htonl(wr->xrc_remote_srq_num << 8); > + /* fall thru */ > case IBV_QPT_RC: > case IBV_QPT_UC: > switch (wr->opcode) { > @@ -543,6 +546,7 @@ void mlx4_calc_sq_wqe_size(struct ibv_qp_cap *cap, enum ibv_qp_type type, > size += sizeof (struct mlx4_wqe_raddr_seg); > break; > > + case IBV_QPT_XRC: > case IBV_QPT_RC: > size += sizeof (struct mlx4_wqe_raddr_seg); > /* > @@ -630,6 +634,7 @@ void mlx4_set_sq_sizes(struct mlx4_qp *qp, struct ibv_qp_cap *cap, > > case IBV_QPT_UC: > case IBV_QPT_RC: > + case IBV_QPT_XRC: > wqe_size -= sizeof (struct mlx4_wqe_raddr_seg); > break; > > diff --git a/src/srq.c b/src/srq.c > index ba2ceb9..1350792 100644 > --- a/src/srq.c > +++ b/src/srq.c > @@ -167,3 +167,53 @@ int mlx4_alloc_srq_buf(struct ibv_pd *pd, struct ibv_srq_attr *attr, > > return 0; > } > + > +struct mlx4_srq *mlx4_find_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn) > +{ > + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; > + > + if (ctx->xrc_srq_table[tind].refcnt) > + return ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask]; > + else > + return NULL; > +} > + > +int mlx4_store_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn, > + struct mlx4_srq *srq) > +{ > + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; > + int ret = 0; > + > + pthread_mutex_lock(&ctx->xrc_srq_table_mutex); > + > + if (!ctx->xrc_srq_table[tind].refcnt) { > + ctx->xrc_srq_table[tind].table = calloc(ctx->xrc_srq_table_mask + 1, > + sizeof(struct mlx4_srq *)); > + if (!ctx->xrc_srq_table[tind].table) { > + ret = -1; > + goto out; > + } > + } > + > + ++ctx->xrc_srq_table[tind].refcnt; > + ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask] = srq; > + > +out: > + pthread_mutex_unlock(&ctx->xrc_srq_table_mutex); > + return ret; > +} > + > +void mlx4_clear_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn) > +{ > + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; > + > + pthread_mutex_lock(&ctx->xrc_srq_table_mutex); > + > + if (!--ctx->xrc_srq_table[tind].refcnt) > + free(ctx->xrc_srq_table[tind].table); > + else > + ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask] = NULL; > + > + pthread_mutex_unlock(&ctx->xrc_srq_table_mutex); > +} > + > diff --git a/src/verbs.c b/src/verbs.c > index 3935ed5..b883992 100644 > --- a/src/verbs.c > +++ b/src/verbs.c > @@ -368,18 +368,36 @@ int mlx4_query_srq(struct ibv_srq *srq, > return ibv_cmd_query_srq(srq, attr, &cmd, sizeof cmd); > } > > -int mlx4_destroy_srq(struct ibv_srq *srq) > +int mlx4_destroy_srq(struct ibv_srq *ibsrq) > { > + struct mlx4_srq *srq = to_msrq(ibsrq); > + struct mlx4_cq *mcq = NULL; > int ret; > > - ret = ibv_cmd_destroy_srq(srq); > - if (ret) > + if (ibsrq->xrc_cq) { > + /* is an xrc_srq */ > + mcq = to_mcq(ibsrq->xrc_cq); > + mlx4_cq_clean(mcq, 0, srq); > + pthread_spin_lock(&mcq->lock); > + mlx4_clear_xrc_srq(to_mctx(ibsrq->context), srq->srqn); > + pthread_spin_unlock(&mcq->lock); > + } > + > + ret = ibv_cmd_destroy_srq(ibsrq); > + if (ret) { > + if (ibsrq->xrc_cq) { > + pthread_spin_lock(&mcq->lock); > + mlx4_store_xrc_srq(to_mctx(ibsrq->context), > + srq->srqn, srq); > + pthread_spin_unlock(&mcq->lock); > + } > return ret; > + } > > - mlx4_free_db(to_mctx(srq->context), MLX4_DB_TYPE_RQ, to_msrq(srq)->db); > - mlx4_free_buf(&to_msrq(srq)->buf); > - free(to_msrq(srq)->wrid); > - free(to_msrq(srq)); > + mlx4_free_db(to_mctx(ibsrq->context), MLX4_DB_TYPE_RQ, srq->db); > + mlx4_free_buf(&srq->buf); > + free(srq->wrid); > + free(srq); > > return 0; > } > @@ -413,7 +431,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) > qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); > qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); > > - if (attr->srq) > + if (attr->srq || attr->qp_type == IBV_QPT_XRC) > attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; > else { > if (attr->cap.max_recv_sge < 1) > @@ -431,7 +449,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) > pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) > goto err_free; > > - if (!attr->srq) { > + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { > qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); > if (!qp->db) > goto err_free; > @@ -440,7 +458,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) > } > > cmd.buf_addr = (uintptr_t) qp->buf.buf; > - if (attr->srq) > + if (attr->srq || attr->qp_type == IBV_QPT_XRC) > cmd.db_addr = 0; > else > cmd.db_addr = (uintptr_t) qp->db; > @@ -477,7 +495,7 @@ err_destroy: > ibv_cmd_destroy_qp(&qp->ibv_qp); > > err_rq_db: > - if (!attr->srq) > + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) > mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); > > err_free: > @@ -536,7 +554,7 @@ int mlx4_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, > mlx4_cq_clean(to_mcq(qp->send_cq), qp->qp_num, NULL); > > mlx4_init_qp_indices(to_mqp(qp)); > - if (!qp->srq) > + if (!qp->srq && qp->qp_type != IBV_QPT_XRC) > *to_mqp(qp)->db = 0; > } > > @@ -595,7 +613,7 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) > > mlx4_unlock_cqs(ibqp); > > - if (!ibqp->srq) > + if (!ibqp->srq && ibqp->qp_type != IBV_QPT_XRC) > mlx4_free_db(to_mctx(ibqp->context), MLX4_DB_TYPE_RQ, qp->db); > free(qp->sq.wrid); > if (qp->rq.wqe_cnt) > @@ -653,3 +671,103 @@ int mlx4_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) > { > return ibv_cmd_detach_mcast(qp, gid, lid); > } > + > +#ifdef HAVE_IBV_XRC_OPS > +struct ibv_srq *mlx4_create_xrc_srq(struct ibv_pd *pd, > + struct ibv_xrc_domain *xrc_domain, > + struct ibv_cq *xrc_cq, > + struct ibv_srq_init_attr *attr) > +{ > + struct mlx4_create_xrc_srq cmd; > + struct mlx4_create_srq_resp resp; > + struct mlx4_srq *srq; > + int ret; > + > + /* Sanity check SRQ size before proceeding */ > + if (attr->attr.max_wr > 1 << 16 || attr->attr.max_sge > 64) > + return NULL; > + > + srq = malloc(sizeof *srq); > + if (!srq) > + return NULL; > + > + if (pthread_spin_init(&srq->lock, PTHREAD_PROCESS_PRIVATE)) > + goto err; > + > + srq->max = align_queue_size(attr->attr.max_wr + 1); > + srq->max_gs = attr->attr.max_sge; > + srq->counter = 0; > + > + if (mlx4_alloc_srq_buf(pd, &attr->attr, srq)) > + goto err; > + > + srq->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); > + if (!srq->db) > + goto err_free; > + > + *srq->db = 0; > + > + cmd.buf_addr = (uintptr_t) srq->buf.buf; > + cmd.db_addr = (uintptr_t) srq->db; > + > + ret = ibv_cmd_create_xrc_srq(pd, &srq->ibv_srq, attr, > + xrc_domain->handle, > + xrc_cq->handle, > + &cmd.ibv_cmd, sizeof cmd, > + &resp.ibv_resp, sizeof resp); > + if (ret) > + goto err_db; > + > + srq->ibv_srq.xrc_srq_num = srq->srqn = resp.srqn; > + > + ret = mlx4_store_xrc_srq(to_mctx(pd->context), srq->ibv_srq.xrc_srq_num, srq); > + if (ret) > + goto err_destroy; > + > + return &srq->ibv_srq; > + > +err_destroy: > + ibv_cmd_destroy_srq(&srq->ibv_srq); > + > +err_db: > + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, srq->db); > + > +err_free: > + free(srq->wrid); > + mlx4_free_buf(&srq->buf); > + > +err: > + free(srq); > + > + return NULL; > +} > + > +struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, > + int fd, int oflag) > +{ > + int ret; > + struct mlx4_open_xrc_domain_resp resp; > + struct mlx4_xrc_domain *xrcd; > + > + xrcd = malloc(sizeof *xrcd); > + if (!xrcd) > + return NULL; > + > + ret = ibv_cmd_open_xrc_domain(context, fd, oflag, &xrcd->ibv_xrcd, > + &resp.ibv_resp, sizeof resp); > + if (ret) { > + free(xrcd); > + return NULL; > + } > + > + xrcd->xrcdn = resp.xrcdn; > + return &xrcd->ibv_xrcd; > +} > + > +int mlx4_close_xrc_domain(struct ibv_xrc_domain *d) > +{ > + ibv_cmd_close_xrc_domain(d); > + free(d); > + return 0; > +} > +#endif > diff --git a/src/wqe.h b/src/wqe.h > index 6f7f309..fa2f8ac 100644 > --- a/src/wqe.h > +++ b/src/wqe.h > @@ -65,7 +65,7 @@ struct mlx4_wqe_ctrl_seg { > * [1] SE (solicited event) > * [0] FL (force loopback) > */ > - uint32_t srcrb_flags; > + uint32_t xrcrb_flags; > /* > * imm is immediate data for send/RDMA write w/ immediate; > * also invalidation key for send with invalidate; input > -- > 1.5.1.6 > > From jackm at dev.mellanox.co.il Mon Jul 14 01:24:44 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 14 Jul 2008 11:24:44 +0300 Subject: [ofa-general] ***SPAM*** Re: [PATCH 3/12 V2] libmlx4: XRC base implementation In-Reply-To: <3b5e77ad0807140059h52fb3ac6g184307b54bf70290@mail.gmail.com> References: <200807101851.30229.jackm@dev.mellanox.co.il> <3b5e77ad0807140059h52fb3ac6g184307b54bf70290@mail.gmail.com> Message-ID: <200807141124.44378.jackm@dev.mellanox.co.il> On Monday 14 July 2008 10:59, Ron Livne wrote: > Hi Jack: > > +#ifdef HAVE_IBV_XRC_OPS > + context->ibv_ctx.more_ops = &mlx4_more_ops; > +#endif > > As you know, I'm using more_ops for my new ibv_create_qp_expanded user verb. > I need the context->ibv_ctx.more_ops to be always initialized, even if > HAVE_IBV_XRC_OPS isn't set. > > I'll release a patch for it. > > Ron Your approach is not correct. The whole point of > > +AC_CHECK_MEMBER(struct ibv_context.more_ops, > > + [AC_DEFINE([HAVE_IBV_MORE_OPS], 1, [Define to 1 if more_ops is a member of ibv_context])],, > > + [#include ]) is to check that the more-ops member exists in the ibv_context structure. It may not exist if an older version of libibverbs is installed on the host, and this possibility must be checked-for (as is done by AC_CHECK_MEMBER above). If more_ops does exist, then the HAVE_IBV_MORE_OPS macro gets defined (at libmlx4 installation time) in config.h, and more_ops will get properly initialized. If it does not exist, then you will not clobber the application by initializing memory which is outside the ibv_context structure. - Jack > On Thu, Jul 10, 2008 at 6:51 PM, Jack Morgenstein > wrote: > > From 33c27150ed20ff34622a45abd5ff27727f4c34aa Mon Sep 17 00:00:00 2001 > > From: Jack Morgenstein > > Date: Wed, 9 Jul 2008 13:55:24 +0300 > > Subject: [PATCH] XRC implementation, consolidated: > > > > xrc ops were moved to their own structure at the end of > > struct ibv_context (to preserve binary compatibility). > > > > Check for ibv_context.xrc_ops member via AC_CHECK_MEMBER > > > > XRC QPs have MSB set in qp number, for identification in > > completion handling. > > > > jackm at mtldesk006:/local/roland/libmlx4> head -n 20 patches/xrc_consolidated.patch > > XRC implementation, consolidated: > > > > xrc ops were moved to their own structure at the end of > > struct ibv_context (to preserve binary compatibility). > > > > Check for ibv_context.xrc_ops member via AC_CHECK_MEMBER > > > > XRC QPs have MSB set in qp number, for identification in > > completion handling. > > > > Various bug fixes. > > (OFED 1.3 commit 39fe7f47e8fc07f356098df048d00740ba585fc5) > > > > V2: > > 1. checkpatch.pl cleanup > > 2. Changed xrc_ops to more ops > > 3. Check for xrc verbs in ibv_more_ops via AC_CHECK_MEMBER > > > > Signed-off-by: Jack Morgenstein > > --- > > configure.in | 3 + > > src/cq.c | 35 ++++++++++++-- > > src/mlx4-abi.h | 16 ++++++ > > src/mlx4.c | 20 ++++++++ > > src/mlx4.h | 46 ++++++++++++++++++ > > src/qp.c | 7 ++- > > src/srq.c | 50 +++++++++++++++++++ > > src/verbs.c | 144 +++++++++++++++++++++++++++++++++++++++++++++++++++----- > > src/wqe.h | 2 +- > > 9 files changed, 303 insertions(+), 20 deletions(-) > > > > diff --git a/configure.in b/configure.in > > index 25f27f7..9304539 100644 > > --- a/configure.in > > +++ b/configure.in > > @@ -42,6 +42,12 @@ AC_CHECK_HEADER(valgrind/memcheck.h, > > dnl Checks for typedefs, structures, and compiler characteristics. > > AC_C_CONST > > AC_CHECK_SIZEOF(long) > > +AC_CHECK_MEMBER(struct ibv_context.more_ops, > > + [AC_DEFINE([HAVE_IBV_MORE_OPS], 1, [Define to 1 if more_ops is a member of ibv_context])],, > > + [#include ]) > > +AC_CHECK_MEMBER(struct ibv_more_ops.create_xrc_srq, > > + [AC_DEFINE([HAVE_IBV_XRC_OPS], 1, [Define to 1 if have xrc ops])],, > > + [#include ]) > > > > dnl Checks for library functions > > AC_CHECK_FUNC(ibv_read_sysfs_file, [], > > diff --git a/src/cq.c b/src/cq.c > > index 68e16e9..c598b87 100644 > > --- a/src/cq.c > > +++ b/src/cq.c > > @@ -196,10 +196,12 @@ static int mlx4_poll_one(struct mlx4_cq *cq, > > struct mlx4_cqe *cqe; > > struct mlx4_srq *srq; > > uint32_t qpn; > > + uint32_t srqn; > > uint32_t g_mlpath_rqpn; > > uint16_t wqe_index; > > int is_error; > > int is_send; > > + int is_src_recv = 0; > > > > cqe = next_cqe_sw(cq); > > if (!cqe) > > @@ -221,20 +223,30 @@ static int mlx4_poll_one(struct mlx4_cq *cq, > > is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == > > MLX4_CQE_OPCODE_ERROR; > > > > - if (!*cur_qp || > > - (ntohl(cqe->my_qpn) & 0xffffff) != (*cur_qp)->ibv_qp.qp_num) { > > + if (qpn & MLX4_XRC_QPN_BIT && !is_send) { > > + srqn = ntohl(cqe->g_mlpath_rqpn) & 0xffffff; > > + /* > > + * We do not have to take the XRC SRQ table lock here, > > + * because CQs will be locked while XRC SRQs are removed > > + * from the table. > > + */ > > + srq = mlx4_find_xrc_srq(to_mctx(cq->ibv_cq.context), srqn); > > + if (!srq) > > + return CQ_POLL_ERR; > > + is_src_recv = 1; > > + } else if (!*cur_qp || (qpn & 0xffffff) != (*cur_qp)->ibv_qp.qp_num) { > > /* > > * We do not have to take the QP table lock here, > > * because CQs will be locked while QPs are removed > > * from the table. > > */ > > *cur_qp = mlx4_find_qp(to_mctx(cq->ibv_cq.context), > > - ntohl(cqe->my_qpn) & 0xffffff); > > + qpn & 0xffffff); > > if (!*cur_qp) > > return CQ_POLL_ERR; > > } > > > > - wc->qp_num = (*cur_qp)->ibv_qp.qp_num; > > + wc->qp_num = qpn & 0xffffff; > > > > if (is_send) { > > wq = &(*cur_qp)->sq; > > @@ -242,6 +254,10 @@ static int mlx4_poll_one(struct mlx4_cq *cq, > > wq->tail += (uint16_t) (wqe_index - (uint16_t) wq->tail); > > wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)]; > > ++wq->tail; > > + } else if (is_src_recv) { > > + wqe_index = htons(cqe->wqe_index); > > + wc->wr_id = srq->wrid[wqe_index]; > > + mlx4_free_srq_wqe(srq, wqe_index); > > } else if ((*cur_qp)->ibv_qp.srq) { > > srq = to_msrq((*cur_qp)->ibv_qp.srq); > > wqe_index = htons(cqe->wqe_index); > > @@ -387,6 +403,10 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) > > uint32_t prod_index; > > uint8_t owner_bit; > > int nfreed = 0; > > + int is_xrc_srq = 0; > > + > > + if (srq && srq->ibv_srq.xrc_cq) > > + is_xrc_srq = 1; > > > > /* > > * First we need to find the current producer index, so we > > @@ -405,7 +425,12 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) > > */ > > while ((int) --prod_index - (int) cq->cons_index >= 0) { > > cqe = get_cqe(cq, prod_index & cq->ibv_cq.cqe); > > - if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { > > + if (is_xrc_srq && > > + (ntohl(cqe->g_mlpath_rqpn & 0xffffff) == srq->srqn) && > > + !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) { > > + mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); > > + ++nfreed; > > + } else if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { > > if (srq && !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) > > mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); > > ++nfreed; > > diff --git a/src/mlx4-abi.h b/src/mlx4-abi.h > > index 20a40c9..1b1253c 100644 > > --- a/src/mlx4-abi.h > > +++ b/src/mlx4-abi.h > > @@ -68,6 +68,14 @@ struct mlx4_resize_cq { > > __u64 buf_addr; > > }; > > > > +#ifdef HAVE_IBV_XRC_OPS > > +struct mlx4_create_xrc_srq { > > + struct ibv_create_xrc_srq ibv_cmd; > > + __u64 buf_addr; > > + __u64 db_addr; > > +}; > > +#endif > > + > > struct mlx4_create_srq { > > struct ibv_create_srq ibv_cmd; > > __u64 buf_addr; > > @@ -90,4 +98,12 @@ struct mlx4_create_qp { > > __u8 reserved[5]; > > }; > > > > +#ifdef HAVE_IBV_XRC_OPS > > +struct mlx4_open_xrc_domain_resp { > > + struct ibv_open_xrc_domain_resp ibv_resp; > > + __u32 xrcdn; > > + __u32 reserved; > > +}; > > +#endif > > + > > #endif /* MLX4_ABI_H */ > > diff --git a/src/mlx4.c b/src/mlx4.c > > index 34ece39..8a46543 100644 > > --- a/src/mlx4.c > > +++ b/src/mlx4.c > > @@ -68,6 +68,16 @@ struct { > > HCA(MELLANOX, 0x673c), /* MT25408 "Hermon" QDR PCIe gen2 */ > > }; > > > > +#ifdef HAVE_IBV_MORE_OPS > > +static struct ibv_more_ops mlx4_more_ops = { > > +#ifdef HAVE_IBV_XRC_OPS > > + .create_xrc_srq = mlx4_create_xrc_srq, > > + .open_xrc_domain = mlx4_open_xrc_domain, > > + .close_xrc_domain = mlx4_close_xrc_domain, > > +#endif > > +}; > > +#endif > > + > > static struct ibv_context_ops mlx4_ctx_ops = { > > .query_device = mlx4_query_device, > > .query_port = mlx4_query_port, > > @@ -123,6 +131,15 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ > > for (i = 0; i < MLX4_QP_TABLE_SIZE; ++i) > > context->qp_table[i].refcnt = 0; > > > > + context->num_xrc_srqs = resp.qp_tab_size; > > + context->xrc_srq_table_shift = ffs(context->num_xrc_srqs) - 1 > > + - MLX4_XRC_SRQ_TABLE_BITS; > > + context->xrc_srq_table_mask = (1 << context->xrc_srq_table_shift) - 1; > > + > > + pthread_mutex_init(&context->xrc_srq_table_mutex, NULL); > > + for (i = 0; i < MLX4_XRC_SRQ_TABLE_SIZE; ++i) > > + context->xrc_srq_table[i].refcnt = 0; > > + > > for (i = 0; i < MLX4_NUM_DB_TYPE; ++i) > > context->db_list[i] = NULL; > > > > @@ -155,6 +172,9 @@ static struct ibv_context *mlx4_alloc_context(struct ibv_device *ibdev, int cmd_ > > pthread_spin_init(&context->uar_lock, PTHREAD_PROCESS_PRIVATE); > > > > context->ibv_ctx.ops = mlx4_ctx_ops; > > +#ifdef HAVE_IBV_XRC_OPS > > + context->ibv_ctx.more_ops = &mlx4_more_ops; > > +#endif > > > > return &context->ibv_ctx; > > > > diff --git a/src/mlx4.h b/src/mlx4.h > > index 827a201..28a8f9c 100644 > > --- a/src/mlx4.h > > +++ b/src/mlx4.h > > @@ -79,6 +79,11 @@ > > > > #endif > > > > +#ifndef HAVE_IBV_MORE_OPS > > +#undef HAVE_IBV_XRC_OPS > > +#undef HAVE_IBV_CREATE_QP_EXP > > +#endif > > + > > #define HIDDEN __attribute__((visibility ("hidden"))) > > > > #define PFX "mlx4: " > > @@ -97,6 +97,16 @@ enum { > > MLX4_QP_TABLE_MASK = MLX4_QP_TABLE_SIZE - 1 > > }; > > > > +enum { > > + MLX4_XRC_SRQ_TABLE_BITS = 8, > > + MLX4_XRC_SRQ_TABLE_SIZE = 1 << MLX4_XRC_SRQ_TABLE_BITS, > > + MLX4_XRC_SRQ_TABLE_MASK = MLX4_XRC_SRQ_TABLE_SIZE - 1 > > +}; > > + > > +enum { > > + MLX4_XRC_QPN_BIT = (1 << 23) > > +}; > > + > > enum mlx4_db_type { > > MLX4_DB_TYPE_CQ, > > MLX4_DB_TYPE_RQ, > > @@ -157,6 +167,15 @@ struct mlx4_context { > > int qp_table_shift; > > int qp_table_mask; > > > > + struct { > > + struct mlx4_srq **table; > > + int refcnt; > > + } xrc_srq_table[MLX4_XRC_SRQ_TABLE_SIZE]; > > + pthread_mutex_t xrc_srq_table_mutex; > > + int num_xrc_srqs; > > + int xrc_srq_table_shift; > > + int xrc_srq_table_mask; > > + > > struct mlx4_db_page *db_list[MLX4_NUM_DB_TYPE]; > > pthread_mutex_t db_list_mutex; > > }; > > @@ -243,6 +262,11 @@ struct mlx4_ah { > > struct mlx4_av av; > > }; > > > > +struct mlx4_xrc_domain { > > + struct ibv_xrc_domain ibv_xrcd; > > + uint32_t xrcdn; > > +}; > > + > > static inline unsigned long align(unsigned long val, unsigned long align) > > { > > return (val + align - 1) & ~(align - 1); > > @@ -287,6 +311,13 @@ static inline struct mlx4_ah *to_mah(struct ibv_ah *ibah) > > return to_mxxx(ah, ah); > > } > > > > +#ifdef HAVE_IBV_XRC_OPS > > +static inline struct mlx4_xrc_domain *to_mxrcd(struct ibv_xrc_domain *ibxrcd) > > +{ > > + return to_mxxx(xrcd, xrc_domain); > > +} > > +#endif > > + > > int mlx4_alloc_buf(struct mlx4_buf *buf, size_t size, int page_size); > > void mlx4_free_buf(struct mlx4_buf *buf); > > > > @@ -333,6 +364,10 @@ void mlx4_free_srq_wqe(struct mlx4_srq *srq, int ind); > > int mlx4_post_srq_recv(struct ibv_srq *ibsrq, > > struct ibv_recv_wr *wr, > > struct ibv_recv_wr **bad_wr); > > +struct mlx4_srq *mlx4_find_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn); > > +int mlx4_store_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn, > > + struct mlx4_srq *srq); > > +void mlx4_clear_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn); > > > > struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr); > > int mlx4_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, > > @@ -363,5 +398,16 @@ int mlx4_alloc_av(struct mlx4_pd *pd, struct ibv_ah_attr *attr, > > void mlx4_free_av(struct mlx4_ah *ah); > > int mlx4_attach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); > > int mlx4_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); > > +#ifdef HAVE_IBV_XRC_OPS > > +struct ibv_srq *mlx4_create_xrc_srq(struct ibv_pd *pd, > > + struct ibv_xrc_domain *xrc_domain, > > + struct ibv_cq *xrc_cq, > > + struct ibv_srq_init_attr *attr); > > +struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, > > + int fd, int oflag); > > + > > +int mlx4_close_xrc_domain(struct ibv_xrc_domain *d); > > +#endif > > + > > > > #endif /* MLX4_H */ > > diff --git a/src/qp.c b/src/qp.c > > index bb98c09..02c5d34 100644 > > --- a/src/qp.c > > +++ b/src/qp.c > > @@ -226,7 +226,7 @@ int mlx4_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, > > ctrl = wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1)); > > qp->sq.wrid[ind & (qp->sq.wqe_cnt - 1)] = wr->wr_id; > > > > - ctrl->srcrb_flags = > > + ctrl->xrcrb_flags = > > (wr->send_flags & IBV_SEND_SIGNALED ? > > htonl(MLX4_WQE_CTRL_CQ_UPDATE) : 0) | > > (wr->send_flags & IBV_SEND_SOLICITED ? > > @@ -243,6 +243,9 @@ int mlx4_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, > > size = sizeof *ctrl / 16; > > > > switch (ibqp->qp_type) { > > + case IBV_QPT_XRC: > > + ctrl->xrcrb_flags |= htonl(wr->xrc_remote_srq_num << 8); > > + /* fall thru */ > > case IBV_QPT_RC: > > case IBV_QPT_UC: > > switch (wr->opcode) { > > @@ -543,6 +546,7 @@ void mlx4_calc_sq_wqe_size(struct ibv_qp_cap *cap, enum ibv_qp_type type, > > size += sizeof (struct mlx4_wqe_raddr_seg); > > break; > > > > + case IBV_QPT_XRC: > > case IBV_QPT_RC: > > size += sizeof (struct mlx4_wqe_raddr_seg); > > /* > > @@ -630,6 +634,7 @@ void mlx4_set_sq_sizes(struct mlx4_qp *qp, struct ibv_qp_cap *cap, > > > > case IBV_QPT_UC: > > case IBV_QPT_RC: > > + case IBV_QPT_XRC: > > wqe_size -= sizeof (struct mlx4_wqe_raddr_seg); > > break; > > > > diff --git a/src/srq.c b/src/srq.c > > index ba2ceb9..1350792 100644 > > --- a/src/srq.c > > +++ b/src/srq.c > > @@ -167,3 +167,53 @@ int mlx4_alloc_srq_buf(struct ibv_pd *pd, struct ibv_srq_attr *attr, > > > > return 0; > > } > > + > > +struct mlx4_srq *mlx4_find_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn) > > +{ > > + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; > > + > > + if (ctx->xrc_srq_table[tind].refcnt) > > + return ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask]; > > + else > > + return NULL; > > +} > > + > > +int mlx4_store_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn, > > + struct mlx4_srq *srq) > > +{ > > + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; > > + int ret = 0; > > + > > + pthread_mutex_lock(&ctx->xrc_srq_table_mutex); > > + > > + if (!ctx->xrc_srq_table[tind].refcnt) { > > + ctx->xrc_srq_table[tind].table = calloc(ctx->xrc_srq_table_mask + 1, > > + sizeof(struct mlx4_srq *)); > > + if (!ctx->xrc_srq_table[tind].table) { > > + ret = -1; > > + goto out; > > + } > > + } > > + > > + ++ctx->xrc_srq_table[tind].refcnt; > > + ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask] = srq; > > + > > +out: > > + pthread_mutex_unlock(&ctx->xrc_srq_table_mutex); > > + return ret; > > +} > > + > > +void mlx4_clear_xrc_srq(struct mlx4_context *ctx, uint32_t xrc_srqn) > > +{ > > + int tind = (xrc_srqn & (ctx->num_xrc_srqs - 1)) >> ctx->xrc_srq_table_shift; > > + > > + pthread_mutex_lock(&ctx->xrc_srq_table_mutex); > > + > > + if (!--ctx->xrc_srq_table[tind].refcnt) > > + free(ctx->xrc_srq_table[tind].table); > > + else > > + ctx->xrc_srq_table[tind].table[xrc_srqn & ctx->xrc_srq_table_mask] = NULL; > > + > > + pthread_mutex_unlock(&ctx->xrc_srq_table_mutex); > > +} > > + > > diff --git a/src/verbs.c b/src/verbs.c > > index 3935ed5..b883992 100644 > > --- a/src/verbs.c > > +++ b/src/verbs.c > > @@ -368,18 +368,36 @@ int mlx4_query_srq(struct ibv_srq *srq, > > return ibv_cmd_query_srq(srq, attr, &cmd, sizeof cmd); > > } > > > > -int mlx4_destroy_srq(struct ibv_srq *srq) > > +int mlx4_destroy_srq(struct ibv_srq *ibsrq) > > { > > + struct mlx4_srq *srq = to_msrq(ibsrq); > > + struct mlx4_cq *mcq = NULL; > > int ret; > > > > - ret = ibv_cmd_destroy_srq(srq); > > - if (ret) > > + if (ibsrq->xrc_cq) { > > + /* is an xrc_srq */ > > + mcq = to_mcq(ibsrq->xrc_cq); > > + mlx4_cq_clean(mcq, 0, srq); > > + pthread_spin_lock(&mcq->lock); > > + mlx4_clear_xrc_srq(to_mctx(ibsrq->context), srq->srqn); > > + pthread_spin_unlock(&mcq->lock); > > + } > > + > > + ret = ibv_cmd_destroy_srq(ibsrq); > > + if (ret) { > > + if (ibsrq->xrc_cq) { > > + pthread_spin_lock(&mcq->lock); > > + mlx4_store_xrc_srq(to_mctx(ibsrq->context), > > + srq->srqn, srq); > > + pthread_spin_unlock(&mcq->lock); > > + } > > return ret; > > + } > > > > - mlx4_free_db(to_mctx(srq->context), MLX4_DB_TYPE_RQ, to_msrq(srq)->db); > > - mlx4_free_buf(&to_msrq(srq)->buf); > > - free(to_msrq(srq)->wrid); > > - free(to_msrq(srq)); > > + mlx4_free_db(to_mctx(ibsrq->context), MLX4_DB_TYPE_RQ, srq->db); > > + mlx4_free_buf(&srq->buf); > > + free(srq->wrid); > > + free(srq); > > > > return 0; > > } > > @@ -413,7 +431,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) > > qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); > > qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); > > > > - if (attr->srq) > > + if (attr->srq || attr->qp_type == IBV_QPT_XRC) > > attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; > > else { > > if (attr->cap.max_recv_sge < 1) > > @@ -431,7 +449,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) > > pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) > > goto err_free; > > > > - if (!attr->srq) { > > + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { > > qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); > > if (!qp->db) > > goto err_free; > > @@ -440,7 +458,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) > > } > > > > cmd.buf_addr = (uintptr_t) qp->buf.buf; > > - if (attr->srq) > > + if (attr->srq || attr->qp_type == IBV_QPT_XRC) > > cmd.db_addr = 0; > > else > > cmd.db_addr = (uintptr_t) qp->db; > > @@ -477,7 +495,7 @@ err_destroy: > > ibv_cmd_destroy_qp(&qp->ibv_qp); > > > > err_rq_db: > > - if (!attr->srq) > > + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) > > mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); > > > > err_free: > > @@ -536,7 +554,7 @@ int mlx4_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, > > mlx4_cq_clean(to_mcq(qp->send_cq), qp->qp_num, NULL); > > > > mlx4_init_qp_indices(to_mqp(qp)); > > - if (!qp->srq) > > + if (!qp->srq && qp->qp_type != IBV_QPT_XRC) > > *to_mqp(qp)->db = 0; > > } > > > > @@ -595,7 +613,7 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) > > > > mlx4_unlock_cqs(ibqp); > > > > - if (!ibqp->srq) > > + if (!ibqp->srq && ibqp->qp_type != IBV_QPT_XRC) > > mlx4_free_db(to_mctx(ibqp->context), MLX4_DB_TYPE_RQ, qp->db); > > free(qp->sq.wrid); > > if (qp->rq.wqe_cnt) > > @@ -653,3 +671,103 @@ int mlx4_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) > > { > > return ibv_cmd_detach_mcast(qp, gid, lid); > > } > > + > > +#ifdef HAVE_IBV_XRC_OPS > > +struct ibv_srq *mlx4_create_xrc_srq(struct ibv_pd *pd, > > + struct ibv_xrc_domain *xrc_domain, > > + struct ibv_cq *xrc_cq, > > + struct ibv_srq_init_attr *attr) > > +{ > > + struct mlx4_create_xrc_srq cmd; > > + struct mlx4_create_srq_resp resp; > > + struct mlx4_srq *srq; > > + int ret; > > + > > + /* Sanity check SRQ size before proceeding */ > > + if (attr->attr.max_wr > 1 << 16 || attr->attr.max_sge > 64) > > + return NULL; > > + > > + srq = malloc(sizeof *srq); > > + if (!srq) > > + return NULL; > > + > > + if (pthread_spin_init(&srq->lock, PTHREAD_PROCESS_PRIVATE)) > > + goto err; > > + > > + srq->max = align_queue_size(attr->attr.max_wr + 1); > > + srq->max_gs = attr->attr.max_sge; > > + srq->counter = 0; > > + > > + if (mlx4_alloc_srq_buf(pd, &attr->attr, srq)) > > + goto err; > > + > > + srq->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); > > + if (!srq->db) > > + goto err_free; > > + > > + *srq->db = 0; > > + > > + cmd.buf_addr = (uintptr_t) srq->buf.buf; > > + cmd.db_addr = (uintptr_t) srq->db; > > + > > + ret = ibv_cmd_create_xrc_srq(pd, &srq->ibv_srq, attr, > > + xrc_domain->handle, > > + xrc_cq->handle, > > + &cmd.ibv_cmd, sizeof cmd, > > + &resp.ibv_resp, sizeof resp); > > + if (ret) > > + goto err_db; > > + > > + srq->ibv_srq.xrc_srq_num = srq->srqn = resp.srqn; > > + > > + ret = mlx4_store_xrc_srq(to_mctx(pd->context), srq->ibv_srq.xrc_srq_num, srq); > > + if (ret) > > + goto err_destroy; > > + > > + return &srq->ibv_srq; > > + > > +err_destroy: > > + ibv_cmd_destroy_srq(&srq->ibv_srq); > > + > > +err_db: > > + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, srq->db); > > + > > +err_free: > > + free(srq->wrid); > > + mlx4_free_buf(&srq->buf); > > + > > +err: > > + free(srq); > > + > > + return NULL; > > +} > > + > > +struct ibv_xrc_domain *mlx4_open_xrc_domain(struct ibv_context *context, > > + int fd, int oflag) > > +{ > > + int ret; > > + struct mlx4_open_xrc_domain_resp resp; > > + struct mlx4_xrc_domain *xrcd; > > + > > + xrcd = malloc(sizeof *xrcd); > > + if (!xrcd) > > + return NULL; > > + > > + ret = ibv_cmd_open_xrc_domain(context, fd, oflag, &xrcd->ibv_xrcd, > > + &resp.ibv_resp, sizeof resp); > > + if (ret) { > > + free(xrcd); > > + return NULL; > > + } > > + > > + xrcd->xrcdn = resp.xrcdn; > > + return &xrcd->ibv_xrcd; > > +} > > + > > +int mlx4_close_xrc_domain(struct ibv_xrc_domain *d) > > +{ > > + ibv_cmd_close_xrc_domain(d); > > + free(d); > > + return 0; > > +} > > +#endif > > diff --git a/src/wqe.h b/src/wqe.h > > index 6f7f309..fa2f8ac 100644 > > --- a/src/wqe.h > > +++ b/src/wqe.h > > @@ -65,7 +65,7 @@ struct mlx4_wqe_ctrl_seg { > > * [1] SE (solicited event) > > * [0] FL (force loopback) > > */ > > - uint32_t srcrb_flags; > > + uint32_t xrcrb_flags; > > /* > > * imm is immediate data for send/RDMA write w/ immediate; > > * also invalidation key for send with invalidate; input > > -- > > 1.5.1.6 > > > > > From eli at dev.mellanox.co.il Mon Jul 14 06:50:09 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 14 Jul 2008 16:50:09 +0300 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: References: Message-ID: <20080714135009.GA29500@mtls03> On Sun, Jul 13, 2008 at 10:16:08PM -0700, Roland Dreier wrote: > > - I merged a decent amount of stuff for IPoIB, including LRO, > "loopback blocking," and better handling of fabric events, and I > don't think I have anything pending. There is this patch that I did not recieve your response for. We think it's reasonable to do this - what do you think? http://lists.openfabrics.org/pipermail/general/2008-July/052460.html > From eli at dev.mellanox.co.il Mon Jul 14 07:08:06 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 14 Jul 2008 17:08:06 +0300 Subject: [ofa-general] IPoIB arps disappearing - LRO / IP forwarding In-Reply-To: <487B38AD.9020809@voltaire.com> References: <20080714111100.GG18983@mtls03> <487B38AD.9020809@voltaire.com> Message-ID: <20080714140806.GI18983@mtls03> On Mon, Jul 14, 2008 at 02:29:49PM +0300, Or Gerlitz wrote: > So with the recent patch to ipoib which adds LRO, should we make the > default being LRO off? or maybe the approach brought by this patch "net: > Disable LRO on devices that are forwarding" > http://marc.info/?l=linux-netdev&m=121389887214425&w=2 would solve the > problem without the user intervention? > The general idea looks good to me. From jon at opengridcomputing.com Mon Jul 14 07:55:07 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Mon, 14 Jul 2008 09:55:07 -0500 Subject: [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807141001.30900.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807091815.25992.olaf.kirch@oracle.com> <20080711163938.GB31816@opengridcomputing.com> <200807141001.30900.olaf.kirch@oracle.com> Message-ID: <20080714145506.GA6578@opengridcomputing.com> On Mon, Jul 14, 2008 at 10:01:29AM +0200, Olaf Kirch wrote: > On Friday 11 July 2008 18:39:39 Jon Mason wrote: > > Changing to the SQD state in iWARP will cause the connection to be torn > > down. Simply ignore this modify_qp calls, and iWARP is happy. > > I guess actually both branches of the if statement could be > skipped for iWARP - there's no point in tweaking the RNR counter > for iWARP, even if it doesn't hurt on cxgb3, right? Yes, setting the RNR is unnecessary for iWARP. > Olaf > > > > > Thanks, > > Jon > > > > Signed-off-by: Jon Mason > > > > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > > index 5835a7e..12a6b0a 100644 > > --- a/net/rds/ib_cm.c > > +++ b/net/rds/ib_cm.c > > @@ -101,7 +101,7 @@ static void rds_ib_connect_complete(struct rds_connection *conn, struct rdma_cm_ > > * so that we learn quickly when our credit accounting is > > * buggy. > > */ > > - if (ic->i_flowctl) { > > + if (ic->i_flowctl && !ic->i_iwarp) { > > /* It seems we have to take a brief detour through SQD state > > * in order to change the RNR retry count. */ > > qp_attr.qp_state = IB_QPS_SQD; > > > > > > -- > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From tziporet at mellanox.co.il Mon Jul 14 08:15:26 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 14 Jul 2008 18:15:26 +0300 Subject: [ofa-general] OFED meeting agenda for today (July 14, 2008) Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> This is the agenda for OFED meeting today (July 14, 2008) 1. OFED 1.4 check readiness for feature freeze Review the list of features: - Kernel base 2.6.27 (start with 2.6.26) - 2.6.26 done, will move to 2.6.27 when rc1 available - Multi-Protocol support, Eth and IB for ConnectX - all patches submitted today - NFS-RDMA - work on kernel 2.6.26 done. Backports not ready yet - iSer Target - Not merged yet - Voltaire please update - New iWARP verbs (fast memory thru send queue, local invalidate send work requests, Read with invalidate, peer-to-peer support in iWCM) - done - IPoIB (LRO and RSS) - LRO - done, RSS - patches to be sent this week - RDS - iWARP support - ongoing - SDP (GA) - patches were sent, merged to OFED tree - Raw QP support with ConnectX - under testing - uDAPL (New socket CM for scalability and interop with Windows, UD extensions) - ongoing - OpenSM - code freeze of openSM can be later since it's a closed user level apps. - ibnetdiscover library - accelerate other tools. - Congestion Control - code in ibutils done; under testing - MVAPICH 1.1 - need update - MVAPICH2 1.2 - need update - OpenMPI 1.3 - need update - anything I missed? 2. Backports status: We miss backports for: - ipath driver - RHEL5, SLES10 and all kernels below 2.6.18 - Qlogic - NFSoRDMA - we need at least RHEL5 and SLES 10 - Jeff Backer - EHCA: SLES10/SP1/SP2 - IBM 3. Open discussion Tziporet From Jeffrey.C.Becker at nasa.gov Mon Jul 14 09:40:02 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Mon, 14 Jul 2008 09:40:02 -0700 Subject: [ofa-general] Insane spam filtering? In-Reply-To: References: Message-ID: <487B8162.2080002@nasa.gov> Hi Roland. I don't know what rule got tripped. However, we are in the process of moving over to a new server, so I'll audit the SPAM rules when we move the mailing lists over. Thanks. -jeff Roland Dreier wrote: > > Your mail to 'general' with the subject > > > > ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 > > > > Is being held until the list moderator can review it for approval. > > > > The reason it is being held: > > > > The message headers matched a filter rule > > Anyone know what's going on here? This same email was sent to > linux-kernel as well, and reached that list with no problem > (http://lkml.org/lkml/2008/7/14/12) and no "***SPAM***" in the subject > line, so I assume something is going haywire in the openfabrics.org mail > server... > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From netsigog_1988 at vegawood.ru Mon Jul 14 09:42:17 2008 From: netsigog_1988 at vegawood.ru (FITZGERALD) Date: Tue, 15 Jul 2008 00:42:17 +0800 Subject: [ofa-general] Even politicians need a day off Message-ID: <20080714164526.83597E60944@openfabrics.org> Global warming declared a hoax by US Senate http://alexandergolu.com/main.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at dev.mellanox.co.il Mon Jul 14 09:42:54 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 14 Jul 2008 19:42:54 +0300 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: References: Message-ID: <487B820E.4020002@mellanox.co.il> Roland Dreier wrote: > Core: > > - I'm waiting to merge the RDMA_CM_EVENT_ADDR_CHANGE changes that > depend on core networking changes until such changes are upstream. > Or, please remind me when that happens. > > - Jack's XRC patch set. I think we're getting closer to converging > here, and I hope to get this merged but we're getting down to the > wire, so we'll see. > I hope we can get those in > HW specific: > > - Yevgeny's mlx4 changes. We'll see how much time is left after I > get done with XRC (which is before this on my list) but to be > honest I'm not sure how mergeable a lot of this is without the > mlx4_en patches that actually use it. > We just posted the mlx4_en patches, and we need to coordinate the merge of them together > - I've been working on memory management extensions support for mlx4, > but I'm not sure if it will be ready in time. Firmware for this > may not be released for a while so it ain't urgent anyway. > We are testing the patches and we already have FW that enable them. I agree its not urgent but it would be good to have it, so ULPs that are interested can be tested over IB too. > Here are a few topics that I believe will not be ready in time for the > 2.6.27 window and will need to wait for 2.6.28 at least: > > - Multiple CQ event vector support. No one has convinced me that we > know how ULPs or userspace apps should decide which vector to use, > and hence little progress has been made since we deferred this > during the 2.6.23 merge window. > We should progress this one even if we missed 2.6.27, especially we need it for RSS, and I know also RDS can gain from it. Tziporet From pmorreale at novell.com Mon Jul 14 09:51:20 2008 From: pmorreale at novell.com (Peter W. Morreale) Date: Mon, 14 Jul 2008 10:51:20 -0600 Subject: [ofa-general] IB verbs client/server programming... Message-ID: <1216054280.15841.28.camel@hermosa.morrealenet> Hello all, I'm starting a new client/server project using RDMA and I'm a little stuck at the moment. I've seen Roland Dreier's slides/client-server example as well as the cmatose.c example in the OFED 1.3 tree. Conceptually, I'm starting to become familiar with channels and PD's and MR's ("Oh my!"), however one thing is not clear to me at all. How do I block waiting for the next RDMA event in my daemon? In other words, what interfaces do I have to look at to implement the "blocking" portion equivalent of a recv(2)? Seems like all the server examples block at rdma_listen(), then run to completion exchanging data. Also, is there any documentation out there on using ib verbs? I looked on the OpenFabrics web site (as well as Googled) and have found little besides the man pages. Thanks in advance for any pointers. -PWM From swise at opengridcomputing.com Mon Jul 14 10:04:39 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 14 Jul 2008 12:04:39 -0500 Subject: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> Message-ID: <487B8727.20003@opengridcomputing.com> Sorry I missed today's call. Status: - NFSRDMA using fastreg: In progress. Should be coded and base testing done this week. - RDS over iWARP: In progress. Still need to debug fastreg usage, but first merge has been done into Olaf's tree. - MEM_MGT_EXTENSIONS - done and merged into roland's tree. Still need backports once we get an ofed-1.4 with 2.6.27-rc1. Note: generic peer-to-peer support in the IWCM was dropped due to lack of resources. Steve. Tziporet Koren wrote: > > This is the agenda for OFED meeting today (July 14, 2008) > > 1. OFED 1.4 check readiness for feature freeze > Review the list of features: > - Kernel base 2.6.27 (start with 2.6.26) - 2.6.26 done, will move to > 2.6.27 when rc1 available > - Multi-Protocol support, Eth and IB for ConnectX - all patches > submitted today > - NFS-RDMA - work on kernel 2.6.26 done. Backports not ready yet > - iSer Target - Not merged yet - Voltaire please update > - New iWARP verbs (fast memory thru send queue, local invalidate > send > work requests, Read with invalidate, peer-to-peer support in iWCM) > - done > - IPoIB (LRO and RSS) - LRO - done, RSS - patches to be sent this > week > - RDS - iWARP support - ongoing > - SDP (GA) - patches were sent, merged to OFED tree > - Raw QP support with ConnectX - under testing > - uDAPL (New socket CM for scalability and interop with Windows, UD > extensions) - ongoing > - OpenSM - code freeze of openSM can be later since it's a closed > user level apps. > - ibnetdiscover library - accelerate other tools. > - Congestion Control - code in ibutils done; under testing > - MVAPICH 1.1 - need update > - MVAPICH2 1.2 - need update > - OpenMPI 1.3 - need update > > - anything I missed? > > 2. Backports status: > We miss backports for: > - ipath driver - RHEL5, SLES10 and all kernels below 2.6.18 - Qlogic > - NFSoRDMA - we need at least RHEL5 and SLES 10 - Jeff Backer > - EHCA: SLES10/SP1/SP2 - IBM > > > 3. Open discussion > > Tziporet > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From jsquyres at cisco.com Mon Jul 14 10:16:25 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Mon, 14 Jul 2008 13:16:25 -0400 Subject: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <487B8727.20003@opengridcomputing.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> <487B8727.20003@opengridcomputing.com> Message-ID: <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> Ditto -- I got pulled in a few different directions and I couldn't make today's call; sorry. - OMPI v1.3 - What is the proposed release schedule for OFED 1.4? - We do not yet have beta tarballs for for OMPI v1.3, but we might be able to get some in time for the OFED v1.4 schedule - Question about IBCM for OFED v1.4: - Any estimate on the following ticket: https://bugs.openfabrics.org/show_bug.cgi?id=963 - Any estimate on the "ibcm_listen() sometimes failing" issue? (I don't think there's a ticket filed against this yet) On Jul 14, 2008, at 1:04 PM, Steve Wise wrote: > Sorry I missed today's call. > > Status: > > - NFSRDMA using fastreg: In progress. Should be coded and base > testing done this week. > > - RDS over iWARP: In progress. Still need to debug fastreg usage, > but first merge has been done into Olaf's tree. > > - MEM_MGT_EXTENSIONS - done and merged into roland's tree. Still > need backports once we get an ofed-1.4 with 2.6.27-rc1. > > Note: generic peer-to-peer support in the IWCM was dropped due to > lack of resources. > > > Steve. > > > Tziporet Koren wrote: >> This is the agenda for OFED meeting today (July 14, 2008) >> >> 1. OFED 1.4 check readiness for feature freeze >> Review the list of features: >> - Kernel base 2.6.27 (start with 2.6.26) - 2.6.26 done, will >> move to >> 2.6.27 when rc1 available >> - Multi-Protocol support, Eth and IB for ConnectX - all patches >> submitted today >> - NFS-RDMA - work on kernel 2.6.26 done. Backports not ready yet >> - iSer Target - Not merged yet - Voltaire please update >> - New iWARP verbs (fast memory thru send queue, local invalidate >> send >> work requests, Read with invalidate, peer-to-peer support in >> iWCM) >> - done >> - IPoIB (LRO and RSS) - LRO - done, RSS - patches to be sent this >> week >> - RDS - iWARP support - ongoing >> - SDP (GA) - patches were sent, merged to OFED tree >> - Raw QP support with ConnectX - under testing >> - uDAPL (New socket CM for scalability and interop with Windows, >> UD >> extensions) - ongoing >> - OpenSM - code freeze of openSM can be later since it's a closed >> user level apps. >> - ibnetdiscover library - accelerate other tools. >> - Congestion Control - code in ibutils done; under testing >> - MVAPICH 1.1 - need update >> - MVAPICH2 1.2 - need update >> - OpenMPI 1.3 - need update >> >> - anything I missed? >> >> 2. Backports status: >> We miss backports for: >> - ipath driver - RHEL5, SLES10 and all kernels below 2.6.18 - >> Qlogic >> - NFSoRDMA - we need at least RHEL5 and SLES 10 - Jeff Backer >> - EHCA: SLES10/SP1/SP2 - IBM >> >> >> 3. Open discussion >> >> Tziporet >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Jeff Squyres Cisco Systems From sean.hefty at intel.com Mon Jul 14 10:28:02 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 14 Jul 2008 10:28:02 -0700 Subject: [ofa-general] cma: add support for TIMEWAIT_EXIT event (second try) In-Reply-To: <5D49E7A8952DC44FB38C38FA0D758EAD14E87F@mtlexch01.mtl.com> References: <48590FF2.8090507@mellanox.co.il> <001201c8dc74$2b8c6f40$b2fc070a@amr.corp.intel.com> <5D49E7A8952DC44FB38C38FA0D758EAD14E87F@mtlexch01.mtl.com> Message-ID: <000301c8e5d6$f7e82f50$9d37170a@amr.corp.intel.com> >Acquiring the handler_mutex explicitly sounds like a good idea. Based on Roland's feedback to a patch from Or, keep your original code with cma_disable_remove() in the if statement. >When will it be commited? Will we have it in 2.6.27? That's up to Roland, but the changes look fine to me. I've already modified the librdmacm to account for this event, so getting it in 2.6.27 would be my preference. Can you re-send your original patch against Roland's for-2.6.27 branch? - Sean From arlin.r.davis at intel.com Mon Jul 14 10:42:24 2008 From: arlin.r.davis at intel.com (Davis, Arlin R) Date: Mon, 14 Jul 2008 10:42:24 -0700 Subject: [ofa-general] RE: [ewg] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> Message-ID: Tziporet, uDAPL socket cm for scalability is done. UD extensions will be done end of week. -arlin >-----Original Message----- >From: ewg-bounces at lists.openfabrics.org >[mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Tziporet Koren >Sent: Monday, July 14, 2008 8:15 AM >To: Tziporet Koren; ewg at lists.openfabrics.org >Cc: general at lists.openfabrics.org >Subject: [ewg] OFED meeting agenda for today (July 14, 2008) > > >This is the agenda for OFED meeting today (July 14, 2008) > >1. OFED 1.4 check readiness for feature freeze > Review the list of features: > - Kernel base 2.6.27 (start with 2.6.26) - 2.6.26 done, >will move to >2.6.27 when rc1 available > - Multi-Protocol support, Eth and IB for ConnectX - all patches >submitted today > - NFS-RDMA - work on kernel 2.6.26 done. Backports not ready yet > - iSer Target - Not merged yet - Voltaire please update > - New iWARP verbs (fast memory thru send queue, local invalidate >send > work requests, Read with invalidate, peer-to-peer >support in iWCM) >- done > - IPoIB (LRO and RSS) - LRO - done, RSS - patches to be sent this >week > - RDS - iWARP support - ongoing > - SDP (GA) - patches were sent, merged to OFED tree > - Raw QP support with ConnectX - under testing > - uDAPL (New socket CM for scalability and interop with Windows, UD >extensions) - ongoing > - OpenSM - code freeze of openSM can be later since it's a closed >user level apps. > - ibnetdiscover library - accelerate other tools. > - Congestion Control - code in ibutils done; under testing > - MVAPICH 1.1 - need update > - MVAPICH2 1.2 - need update > - OpenMPI 1.3 - need update > > - anything I missed? > >2. Backports status: > We miss backports for: > - ipath driver - RHEL5, SLES10 and all kernels below 2.6.18 - Qlogic > - NFSoRDMA - we need at least RHEL5 and SLES 10 - Jeff Backer > - EHCA: SLES10/SP1/SP2 - IBM > > >3. Open discussion > >Tziporet > >_______________________________________________ >ewg mailing list >ewg at lists.openfabrics.org >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > From nab at linux-iscsi.org Mon Jul 14 11:02:13 2008 From: nab at linux-iscsi.org (Nicholas A. Bellinger) Date: Mon, 14 Jul 2008 11:02:13 -0700 Subject: [ofa-general] VHACS-VM x86_64 Alpha Preview on 45nm multicore Message-ID: <1216058533.21408.83.camel@haakon2.linux-iscsi.org> Greetings all, I am pleased to announce the Alpha availibility of VHACS-VM Demo for Hardware Virtualized x86_64 running (initially) on VMWare Workstation 6. The VHACS cluster stack also obviously runs on 'bare-metal' hardware, but today VHACS-VM is the easiest method to get your own open storage cloud up and running. What is VHACS..? VHACS is a Cloud Storage implementation running on Linux v2.6. VHACS is an acrynom for Virtualization, High Availibility, and Cluster Storage. VHACS is a combination of at least eight (8) long term OSS/Linux based projects, along with a CLI management interface for controlling VHACS nodes, clouds, and vservers within the VHACS cluster. Here is a quick rundown of the projects (so far) that make up the VHACS cluster stack: CLUSTER: * ) Pacemaker The scalable High-Availability cluster resource manager formerly part of Heartbeat * ) OpenAIS The OpenAIS Standards Based Cluster Framework is an OSI Certified implementation of the Service Availability Forum Application Interface Specification (AIS) SERVER: * ) LIO-Target w/ IBLOCK LIO-Core subsystem plugin for iSCSI Target mode export * ) DRBD Distributed Replicated Block device for Network RAID 1 Mirrors * ) Linux Logical Volume Manager (LVM) ('vhacs' volume group for creation of VHACS clouds) CLIENT: * ) Open/iSCSI iSCSI Initiator client * ) Ext3 filesystem * ) Linux-Vserver Linux Virtualization Layer I am happy to report that VHACS-VM Alpha Demo x86_64 is running 0.8.15 VHACS code, along with the many other pieces of open source infrastructure that the VHACS cloud builds upon, and providing Active/Active H/A with Synchronous Data Replication iSCSI Target ERL=2 / MC/2 export that makes up the SERVER side of the VHACS cloud. The CLIENT side of the VHACS cloud is also running an iSCSI Initiator ext3 mounted filesystem with Open/iSCSI on top of Intel 45 nanometer 'Wolfdale' microprocessor silicon and DDR2 800 memory. Futhermore, using commerically available x86 virtualization technology on open/closed platforms, I am very happy to report that VHACS-VM 2x node clusters are now up and running across both OPEN (Linux x86_64) and CLOSED (win32) x86 platforms on top of the 45nm chips with the VMMU/VMX parts enabled. The first OS independent storage clouds running on top of v2.6.25.9-kdb are now a reality. Here is a link to the code, screenshots, and all the goodies: http://linux-iscsi.org/index.php/VHACS-VM http://linux-iscsi.org/index.php/VHACS The VHACS-VM x86_64 system images, which contain a Debian Etch system iamges running the very latest compontents that make up the VHACS cluster stack. Also included in the vhac64-west and vhacs64-east system images is a complete development environment, including a KDB enabled kernel that will allow the developer to pause the running image at any time, examine memory, or setup a breakpoint to track down a problem. The x86_64 system images currently require hardware x86 virtualization, please see the wiki for more information about hardware requirements. They are available from: http://linux-iscsi.org/builds/VHACS-VM/x86_64/vmware6/ Also, there is great interest to see VHACS-VM running on KVM and QEMU Virtualization platforms. This is something that should not be too difficult as long as the virtualization platform supports execution of x86_64 system images. i386 system images are also in the works to bring the storage cloud to legacy environments without hardware x86_64 virtualization. Enjoy, --nab From fenkes at de.ibm.com Mon Jul 14 12:28:53 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 14 Jul 2008 21:28:53 +0200 Subject: [ofa-general] [PATCH 0/5] eHCA backports for OFED 1.4, part 1 Message-ID: <200807142128.53998.fenkes@de.ibm.com> Hi, this patchset contains eHCA backports for several kernels as well as some fixes to other backports that broke on powerpc. [1/5] fixes core for 2.6.24 [2/5] fixes core for RHEL5.1 and RHEL 5.2 [3/5] adds ehca backports for 2.6.2[345] [4/5] adds ehca backports for RHEL5.1 and RHEL5.2 [5/5] adds ehca backports for SLES10 SP1 and SP2 Please review the changes, especially the core changes, and apply if you think they're okay. Thanks and regards! Joachim and Alex From fenkes at de.ibm.com Mon Jul 14 12:30:22 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 14 Jul 2008 21:30:22 +0200 Subject: [ofa-general] [PATCH 1/5] IB/core: fix kobject.h backport for 2.6.24 In-Reply-To: <200807142128.53998.fenkes@de.ibm.com> References: <200807142128.53998.fenkes@de.ibm.com> Message-ID: <200807142130.22697.fenkes@de.ibm.com> From: Alexander Schmidt The backport kobject.h was causing an include recursion (included slab.h which in turn needed kobject.h), so move the function bodies to their own .c file in core, which is the only place where the new functions are needed. Signed-off-by: Joachim Fenkes --- .../backport/2.6.24/include/linux/kobject.h | 134 +--------------- .../backport/2.6.24/core_1_kobject_backport.patch | 175 ++++++++++++++++++++ 2 files changed, 178 insertions(+), 131 deletions(-) create mode 100644 kernel_patches/backport/2.6.24/core_1_kobject_backport.patch diff --git a/kernel_addons/backport/2.6.24/include/linux/kobject.h b/kernel_addons/backport/2.6.24/include/linux/kobject.h index 3b842f4..f23d141 100644 --- a/kernel_addons/backport/2.6.24/include/linux/kobject.h +++ b/kernel_addons/backport/2.6.24/include/linux/kobject.h @@ -1,57 +1,8 @@ #ifndef __BACKPORT_KOBJECT_H_TO_2_6_24__ #define __BACKPORT_KOBJECT_H_TO_2_6_24__ -#include #include_next -struct kobj_attribute { - struct attribute attr; - ssize_t (*show)(struct kobject *kobj, struct kobj_attribute *attr, - char *buf); - ssize_t (*store)(struct kobject *kobj, struct kobj_attribute *attr, - const char *buf, size_t count); -}; - -/* default kobject attribute operations */ -static ssize_t kobj_attr_show(struct kobject *kobj, struct attribute *attr, - char *buf) -{ - struct kobj_attribute *kattr; - ssize_t ret = -EIO; - - kattr = container_of(attr, struct kobj_attribute, attr); - if (kattr->show) - ret = kattr->show(kobj, kattr, buf); - return ret; -} - -static ssize_t kobj_attr_store(struct kobject *kobj, struct attribute *attr, - const char *buf, size_t count) -{ - struct kobj_attribute *kattr; - ssize_t ret = -EIO; - - kattr = container_of(attr, struct kobj_attribute, attr); - if (kattr->store) - ret = kattr->store(kobj, kattr, buf, count); - return ret; -} - -static struct sysfs_ops kobj_sysfs_ops = { - .show = kobj_attr_show, - .store = kobj_attr_store, -}; - -static void dynamic_kobj_release(struct kobject *kobj) -{ - pr_debug("kobject: (%p): %s\n", kobj, __FUNCTION__); - kfree(kobj); -} - -static struct kobj_type dynamic_kobj_ktype = { - .release = dynamic_kobj_release, - .sysfs_ops = &kobj_sysfs_ops, -}; /** * kobject_create_and_add - create a struct kobject dynamically and register it with sysfs @@ -66,39 +17,7 @@ static struct kobj_type dynamic_kobj_ktype = { * * If the kobject was not able to be created, NULL will be returned. */ -static inline struct kobject *kobject_create_and_add(const char *name, struct kobject *parent) -{ - struct kobject *kobj; - int retval; - - kobj = kzalloc(sizeof(*kobj), GFP_KERNEL); - if (!kobj) - return NULL; - - kobject_init(kobj); - kobj->ktype = &dynamic_kobj_ktype; - kobj->parent = parent; - - retval = kobject_set_name(kobj, "%s", name); - if (retval) { - printk(KERN_WARNING "%s: kobject_set_name error: %d\n", - __FUNCTION__, retval); - goto err; - } - - retval = kobject_add(kobj); - if (retval) { - printk(KERN_WARNING "%s: kobject_add error: %d\n", - __FUNCTION__, retval); - goto err; - } - - return kobj; - -err: - kobject_put(kobj); - return NULL; -} +struct kobject *kobject_create_and_add(const char *name, struct kobject *parent); /** * kobject_init_and_add - initialize a kobject structure and add it to the kobject hierarchy @@ -111,54 +30,7 @@ err: * kobject_add(). The same type of error handling after a call to * kobject_add() and kobject lifetime rules are the same here. */ -static int kobject_init_and_add(struct kobject *kobj, struct kobj_type *ktype, - struct kobject *parent, const char *fmt, ...) -{ - int retval; - int limit; - int need; - va_list args; - char *name; - - /* find out how big a buffer we need */ - name = kmalloc(1024, GFP_KERNEL); - if (!name) { - retval = -ENOMEM; - goto out; - } - va_start(args, fmt); - need = vsnprintf(name, 1024, fmt, args); - va_end(args); - kfree(name); - - /* Allocate the new space and copy the string in */ - limit = need + 1; - name = kmalloc(limit, GFP_KERNEL); - if (!name) { - retval = -ENOMEM; - goto out; - } - - va_start(args, fmt); - need = vsnprintf(name, limit, fmt, args); - va_end(args); - - kobject_init(kobj); - - kobj->ktype = ktype; - kobj->parent = parent; - - retval = kobject_set_name(kobj, name); - kfree(name); - if (retval) - goto out; - - retval = kobject_add(kobj); - if (retval) - goto out; - -out: - return retval; -} +int kobject_init_and_add(struct kobject *kobj, struct kobj_type *ktype, + struct kobject *parent, const char *fmt, ...); #endif /* __BACKPORT_KOBJECT_H_TO_2_6_24__ */ diff --git a/kernel_patches/backport/2.6.24/core_1_kobject_backport.patch b/kernel_patches/backport/2.6.24/core_1_kobject_backport.patch new file mode 100644 index 0000000..fa770e3 --- /dev/null +++ b/kernel_patches/backport/2.6.24/core_1_kobject_backport.patch @@ -0,0 +1,175 @@ +diff -Nurp ofa_kernel-1.4-orig/drivers/infiniband/core/kobject_backport.c ofa_kernel-1.4/drivers/infiniband/core/kobject_backport.c +--- ofa_kernel-1.4-orig/drivers/infiniband/core/kobject_backport.c 1970-01-01 01:00:00.000000000 +0100 ++++ ofa_kernel-1.4/drivers/infiniband/core/kobject_backport.c 2008-07-11 19:52:38.000000000 +0200 +@@ -0,0 +1,159 @@ ++#include ++ ++struct kobj_attribute { ++ struct attribute attr; ++ ssize_t (*show)(struct kobject *kobj, struct kobj_attribute *attr, ++ char *buf); ++ ssize_t (*store)(struct kobject *kobj, struct kobj_attribute *attr, ++ const char *buf, size_t count); ++}; ++ ++/* default kobject attribute operations */ ++static ssize_t kobj_attr_show(struct kobject *kobj, struct attribute *attr, ++ char *buf) ++{ ++ struct kobj_attribute *kattr; ++ ssize_t ret = -EIO; ++ ++ kattr = container_of(attr, struct kobj_attribute, attr); ++ if (kattr->show) ++ ret = kattr->show(kobj, kattr, buf); ++ return ret; ++} ++ ++static ssize_t kobj_attr_store(struct kobject *kobj, struct attribute *attr, ++ const char *buf, size_t count) ++{ ++ struct kobj_attribute *kattr; ++ ssize_t ret = -EIO; ++ ++ kattr = container_of(attr, struct kobj_attribute, attr); ++ if (kattr->store) ++ ret = kattr->store(kobj, kattr, buf, count); ++ return ret; ++} ++ ++static struct sysfs_ops kobj_sysfs_ops = { ++ .show = kobj_attr_show, ++ .store = kobj_attr_store, ++}; ++ ++static void dynamic_kobj_release(struct kobject *kobj) ++{ ++ pr_debug("kobject: (%p): %s\n", kobj, __FUNCTION__); ++ kfree(kobj); ++} ++ ++static struct kobj_type dynamic_kobj_ktype = { ++ .release = dynamic_kobj_release, ++ .sysfs_ops = &kobj_sysfs_ops, ++}; ++ ++/** ++ * kobject_create_and_add - create a struct kobject dynamically and register it with sysfs ++ * ++ * @name: the name for the kset ++ * @parent: the parent kobject of this kobject, if any. ++ * ++ * This function creates a kobject structure dynamically and registers it ++ * with sysfs. When you are finished with this structure, call ++ * kobject_put() and the structure will be dynamically freed when ++ * it is no longer being used. ++ * ++ * If the kobject was not able to be created, NULL will be returned. ++ */ ++struct kobject *kobject_create_and_add(const char *name, struct kobject *parent) ++{ ++ struct kobject *kobj; ++ int retval; ++ ++ kobj = kzalloc(sizeof(*kobj), GFP_KERNEL); ++ if (!kobj) ++ return NULL; ++ ++ kobject_init(kobj); ++ kobj->ktype = &dynamic_kobj_ktype; ++ kobj->parent = parent; ++ ++ retval = kobject_set_name(kobj, "%s", name); ++ if (retval) { ++ printk(KERN_WARNING "%s: kobject_set_name error: %d\n", ++ __FUNCTION__, retval); ++ goto err; ++ } ++ ++ retval = kobject_add(kobj); ++ if (retval) { ++ printk(KERN_WARNING "%s: kobject_add error: %d\n", ++ __FUNCTION__, retval); ++ goto err; ++ } ++ ++ return kobj; ++ ++err: ++ kobject_put(kobj); ++ return NULL; ++} ++ ++/** ++ * kobject_init_and_add - initialize a kobject structure and add it to the kobject hierarchy ++ * @kobj: pointer to the kobject to initialize ++ * @ktype: pointer to the ktype for this kobject. ++ * @parent: pointer to the parent of this kobject. ++ * @fmt: the name of the kobject. ++ * ++ * This function combines the call to kobject_init() and ++ * kobject_add(). The same type of error handling after a call to ++ * kobject_add() and kobject lifetime rules are the same here. ++ */ ++int kobject_init_and_add(struct kobject *kobj, struct kobj_type *ktype, ++ struct kobject *parent, const char *fmt, ...) ++{ ++ int retval; ++ int limit; ++ int need; ++ va_list args; ++ char *name; ++ ++ /* find out how big a buffer we need */ ++ name = kmalloc(1024, GFP_KERNEL); ++ if (!name) { ++ retval = -ENOMEM; ++ goto out; ++ } ++ va_start(args, fmt); ++ need = vsnprintf(name, 1024, fmt, args); ++ va_end(args); ++ kfree(name); ++ ++ /* Allocate the new space and copy the string in */ ++ limit = need + 1; ++ name = kmalloc(limit, GFP_KERNEL); ++ if (!name) { ++ retval = -ENOMEM; ++ goto out; ++ } ++ ++ va_start(args, fmt); ++ need = vsnprintf(name, limit, fmt, args); ++ va_end(args); ++ ++ kobject_init(kobj); ++ ++ kobj->ktype = ktype; ++ kobj->parent = parent; ++ ++ retval = kobject_set_name(kobj, name); ++ kfree(name); ++ if (retval) ++ goto out; ++ ++ retval = kobject_add(kobj); ++ if (retval) ++ goto out; ++ ++out: ++ return retval; ++} ++ +diff -Nurp ofa_kernel-1.4-orig/drivers/infiniband/core/Makefile ofa_kernel-1.4/drivers/infiniband/core/Makefile +--- ofa_kernel-1.4-orig/drivers/infiniband/core/Makefile 2008-07-07 15:06:10.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/core/Makefile 2008-07-11 20:01:44.000000000 +0200 +@@ -8,7 +8,7 @@ obj-$(CONFIG_INFINIBAND_USER_ACCESS) += + $(user_access-y) + + ib_core-y := packer.o ud_header.o verbs.o sysfs.o \ +- device.o fmr_pool.o cache.o ++ device.o fmr_pool.o cache.o kobject_backport.o + ib_core-$(CONFIG_INFINIBAND_USER_MEM) += umem.o + + ib_mad-y := mad.o smi.o agent.o mad_rmpp.o -- 1.5.5 From fenkes at de.ibm.com Mon Jul 14 12:30:56 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 14 Jul 2008 21:30:56 +0200 Subject: [ofa-general] [PATCH 2/5] IB/core: PowerPC backport fixes for RH5.* kernels In-Reply-To: <200807142128.53998.fenkes@de.ibm.com> References: <200807142128.53998.fenkes@de.ibm.com> Message-ID: <200807142130.56881.fenkes@de.ibm.com> From: Alexander Schmidt Signed-off-by: Joachim Fenkes --- .../2.6.18-EL5.1/include/linux/dma-mapping.h | 2 +- .../2.6.18-EL5.2/include/linux/dma-mapping.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel_addons/backport/2.6.18-EL5.1/include/linux/dma-mapping.h b/kernel_addons/backport/2.6.18-EL5.1/include/linux/dma-mapping.h index f219b21..7fd3254 100644 --- a/kernel_addons/backport/2.6.18-EL5.1/include/linux/dma-mapping.h +++ b/kernel_addons/backport/2.6.18-EL5.1/include/linux/dma-mapping.h @@ -3,7 +3,7 @@ #include_next -#ifndef __x86_64__ +#if !defined(__x86_64__) && !defined(__powerpc__) /* x86_64 was the only architecture that had valid_dma_direction * before 2.6.18 */ diff --git a/kernel_addons/backport/2.6.18-EL5.2/include/linux/dma-mapping.h b/kernel_addons/backport/2.6.18-EL5.2/include/linux/dma-mapping.h index f219b21..7fd3254 100644 --- a/kernel_addons/backport/2.6.18-EL5.2/include/linux/dma-mapping.h +++ b/kernel_addons/backport/2.6.18-EL5.2/include/linux/dma-mapping.h @@ -3,7 +3,7 @@ #include_next -#ifndef __x86_64__ +#if !defined(__x86_64__) && !defined(__powerpc__) /* x86_64 was the only architecture that had valid_dma_direction * before 2.6.18 */ -- 1.5.5 From fenkes at de.ibm.com Mon Jul 14 12:32:31 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 14 Jul 2008 21:32:31 +0200 Subject: [ofa-general] [PATCH 3/5] IB/ehca: 2.6.2[345] backports In-Reply-To: <200807142128.53998.fenkes@de.ibm.com> References: <200807142128.53998.fenkes@de.ibm.com> Message-ID: <200807142132.32040.fenkes@de.ibm.com> From: Alexander Schmidt Signed-off-by: Joachim Fenkes --- .../backport/2.6.23/ehca_01_ibmebus_loc_code.patch | 186 ++++++++++++++++++++ .../backport/2.6.24/ehca_01_sysfs_groups.patch | 45 +++++ 2 files changed, 231 insertions(+), 0 deletions(-) create mode 100644 kernel_patches/backport/2.6.23/ehca_01_ibmebus_loc_code.patch create mode 100644 kernel_patches/backport/2.6.24/ehca_01_sysfs_groups.patch diff --git a/kernel_patches/backport/2.6.23/ehca_01_ibmebus_loc_code.patch b/kernel_patches/backport/2.6.23/ehca_01_ibmebus_loc_code.patch new file mode 100644 index 0000000..8737b79 --- /dev/null +++ b/kernel_patches/backport/2.6.23/ehca_01_ibmebus_loc_code.patch @@ -0,0 +1,186 @@ +diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h +index 1e9e99a..fbf6e58 100644 +--- a/drivers/infiniband/hw/ehca/ehca_classes.h ++++ b/drivers/infiniband/hw/ehca/ehca_classes.h +@@ -112,7 +112,7 @@ struct ehca_sport { + + struct ehca_shca { + struct ib_device ib_device; +- struct of_device *ofdev; ++ struct ibmebus_dev *ibmebus_dev; + u8 num_ports; + int hw_level; + struct list_head shca_list; +diff --git a/drivers/infiniband/hw/ehca/ehca_eq.c b/drivers/infiniband/hw/ehca/ehca_eq.c +index 49660df..aab80e9 100644 +--- a/drivers/infiniband/hw/ehca/ehca_eq.c ++++ b/drivers/infiniband/hw/ehca/ehca_eq.c +@@ -122,7 +122,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + /* register interrupt handlers and initialize work queues */ + if (type == EHCA_EQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_eq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_eq, + IRQF_DISABLED, "ehca_eq", + (void *)shca); + if (ret < 0) +@@ -130,7 +130,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + tasklet_init(&eq->interrupt_task, ehca_tasklet_eq, (long)shca); + } else if (type == EHCA_NEQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_neq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_neq, + IRQF_DISABLED, "ehca_neq", + (void *)shca); + if (ret < 0) +@@ -170,7 +170,7 @@ int ehca_destroy_eq(struct ehca_shca *shca, struct ehca_eq *eq) + u64 h_ret; + + spin_lock_irqsave(&eq->spinlock, flags); +- ibmebus_free_irq(eq->ist, (void *)shca); ++ ibmebus_free_irq(NULL, eq->ist, (void *)shca); + + h_ret = hipz_h_destroy_eq(shca->ipz_hca_handle, eq); + +diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c +index 482103e..666aafe 100644 +--- a/drivers/infiniband/hw/ehca/ehca_main.c ++++ b/drivers/infiniband/hw/ehca/ehca_main.c +@@ -289,8 +289,8 @@ static int ehca_sense_attributes(struct + }; + + ehca_gen_dbg("Probing adapter %s...", +- shca->ofdev->node->full_name); +- loc_code = of_get_property(shca->ofdev->node, "ibm,loc-code", NULL); ++ shca->ibmebus_dev->ofdev.node->full_name); ++ loc_code = of_get_property(shca->ibmebus_dev->ofdev.node, "ibm,loc-code", NULL); + if (loc_code) + ehca_gen_dbg(" ... location lode=%s", loc_code); + +@@ -458,7 +458,7 @@ static int ehca_init_device(struct ehca_shca *shca) + shca->ib_device.node_type = RDMA_NODE_IB_CA; + shca->ib_device.phys_port_cnt = shca->num_ports; + shca->ib_device.num_comp_vectors = 1; +- shca->ib_device.dma_device = &shca->ofdev->dev; ++ shca->ib_device.dma_device = &shca->ibmebus_dev->ofdev.dev; + shca->ib_device.query_device = ehca_query_device; + shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_gid = ehca_query_gid; +@@ -619,11 +619,6 @@ static struct attribute_group ehca_drv_attr_grp = { + .attrs = ehca_drv_attrs + }; + +-static struct attribute_group *ehca_drv_attr_groups[] = { +- &ehca_drv_attr_grp, +- NULL, +-}; +- + #define EHCA_RESOURCE_ATTR(name) \ + static ssize_t ehca_show_##name(struct device *dev, \ + struct device_attribute *attr, \ +@@ -707,7 +702,7 @@ static struct attribute_group ehca_dev_attr_grp = { + .attrs = ehca_dev_attrs + }; + +-static int __devinit ehca_probe(struct of_device *dev, ++static int __devinit ehca_probe(struct ibmebus_dev *dev, + const struct of_device_id *id) + { + struct ehca_shca *shca; +@@ -715,16 +710,16 @@ static int __devinit ehca_probe(struct of_device *dev, + struct ib_pd *ibpd; + int ret, i, eq_size; + +- handle = of_get_property(dev->node, "ibm,hca-handle", NULL); ++ handle = of_get_property(dev->ofdev.node, "ibm,hca-handle", NULL); + if (!handle) { + ehca_gen_err("Cannot get eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + + if (!(*handle)) { + ehca_gen_err("Wrong eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + +@@ -739,9 +734,9 @@ static int __devinit ehca_probe(struct of_device *dev, + for (i = 0; i < ARRAY_SIZE(shca->sport); i++) + spin_lock_init(&shca->sport[i].mod_sqp_lock); + +- shca->ofdev = dev; ++ shca->ibmebus_dev = dev; + shca->ipz_hca_handle.handle = *handle; +- dev->dev.driver_data = shca; ++ dev->ofdev.dev.driver_data = shca; + + ret = ehca_sense_attributes(shca); + if (ret < 0) { +@@ -818,7 +813,7 @@ static int __devinit ehca_probe(struct of_device *dev, + } + } + +- ret = sysfs_create_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ ret = sysfs_create_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + if (ret) /* only complain; we can live without attributes */ + ehca_err(&shca->ib_device, + "Cannot create device attributes ret=%d", ret); +@@ -868,12 +863,12 @@ probe1: + return -EINVAL; + } + +-static int __devexit ehca_remove(struct of_device *dev) ++static int __devexit ehca_remove(struct ibmebus_dev *dev) + { +- struct ehca_shca *shca = dev->dev.driver_data; ++ struct ehca_shca *shca = dev->ofdev.dev.driver_data; + int ret; + +- sysfs_remove_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ sysfs_remove_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + + if (ehca_open_aqp1 == 1) { + int i; +@@ -924,14 +919,11 @@ static struct of_device_id ehca_device_table[] = + {}, + }; + +-static struct of_platform_driver ehca_driver = { +- .name = "ehca", +- .match_table = ehca_device_table, +- .probe = ehca_probe, +- .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, ++static struct ibmebus_driver ehca_driver = { ++ .name = "ehca", ++ .id_table = ehca_device_table, ++ .probe = ehca_probe, ++ .remove = ehca_remove, + }; + + void ehca_poll_eqs(unsigned long data) +@@ -990,6 +982,10 @@ static int __init ehca_module_init(void) + goto module_init2; + } + ++ ret = sysfs_create_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); ++ if (ret) /* only complain; we can live without attributes */ ++ ehca_gen_err("Cannot create driver attributes ret=%d", ret); ++ + if (ehca_poll_all_eqs != 1) { + ehca_gen_err("WARNING!!!"); + ehca_gen_err("It is possible to lose interrupts."); +@@ -1015,6 +1011,7 @@ static void __exit ehca_module_exit(void) + if (ehca_poll_all_eqs == 1) + del_timer_sync(&poll_eqs_timer); + ++ sysfs_remove_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); + ibmebus_unregister_driver(&ehca_driver); + + ehca_destroy_slab_caches(); + diff --git a/kernel_patches/backport/2.6.24/ehca_01_sysfs_groups.patch b/kernel_patches/backport/2.6.24/ehca_01_sysfs_groups.patch new file mode 100644 index 0000000..a5cd9bc --- /dev/null +++ b/kernel_patches/backport/2.6.24/ehca_01_sysfs_groups.patch @@ -0,0 +1,45 @@ +diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c +index 482103e..6169670 100644 +--- a/drivers/infiniband/hw/ehca/ehca_main.c ++++ b/drivers/infiniband/hw/ehca/ehca_main.c +@@ -619,11 +619,6 @@ static struct attribute_group ehca_drv_attr_grp = { + .attrs = ehca_drv_attrs + }; + +-static struct attribute_group *ehca_drv_attr_groups[] = { +- &ehca_drv_attr_grp, +- NULL, +-}; +- + #define EHCA_RESOURCE_ATTR(name) \ + static ssize_t ehca_show_##name(struct device *dev, \ + struct device_attribute *attr, \ +@@ -929,9 +924,6 @@ static struct of_platform_driver ehca_driver = { + .match_table = ehca_device_table, + .probe = ehca_probe, + .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, + }; + + void ehca_poll_eqs(unsigned long data) +@@ -990,6 +982,10 @@ static int __init ehca_module_init(void) + goto module_init2; + } + ++ ret = sysfs_create_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); ++ if (ret) /* only complain; we can live without attributes */ ++ ehca_gen_err("Cannot create driver attributes ret=%d", ret); ++ + if (ehca_poll_all_eqs != 1) { + ehca_gen_err("WARNING!!!"); + ehca_gen_err("It is possible to lose interrupts."); +@@ -1015,6 +1011,7 @@ static void __exit ehca_module_exit(void) + if (ehca_poll_all_eqs == 1) + del_timer_sync(&poll_eqs_timer); + ++ sysfs_remove_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); + ibmebus_unregister_driver(&ehca_driver); + + ehca_destroy_slab_caches(); -- 1.5.5 From fenkes at de.ibm.com Mon Jul 14 12:33:11 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 14 Jul 2008 21:33:11 +0200 Subject: [ofa-general] [PATCH 4/5] IB/ehca: RHEL5 backports In-Reply-To: <200807142128.53998.fenkes@de.ibm.com> References: <200807142128.53998.fenkes@de.ibm.com> Message-ID: <200807142133.11922.fenkes@de.ibm.com> From: Alexander Schmidt Signed-off-by: Joachim Fenkes --- .../2.6.18-EL5.1/ehca-01-ibmebus_loc_code.patch | 186 ++++++++++++++++++++ .../2.6.18-EL5.2/ehca-01-ibmebus_loc_code.patch | 186 ++++++++++++++++++++ 2 files changed, 372 insertions(+), 0 deletions(-) create mode 100644 kernel_patches/backport/2.6.18-EL5.1/ehca-01-ibmebus_loc_code.patch create mode 100644 kernel_patches/backport/2.6.18-EL5.2/ehca-01-ibmebus_loc_code.patch diff --git a/kernel_patches/backport/2.6.18-EL5.1/ehca-01-ibmebus_loc_code.patch b/kernel_patches/backport/2.6.18-EL5.1/ehca-01-ibmebus_loc_code.patch new file mode 100644 index 0000000..8737b79 --- /dev/null +++ b/kernel_patches/backport/2.6.18-EL5.1/ehca-01-ibmebus_loc_code.patch @@ -0,0 +1,186 @@ +diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h +index 1e9e99a..fbf6e58 100644 +--- a/drivers/infiniband/hw/ehca/ehca_classes.h ++++ b/drivers/infiniband/hw/ehca/ehca_classes.h +@@ -112,7 +112,7 @@ struct ehca_sport { + + struct ehca_shca { + struct ib_device ib_device; +- struct of_device *ofdev; ++ struct ibmebus_dev *ibmebus_dev; + u8 num_ports; + int hw_level; + struct list_head shca_list; +diff --git a/drivers/infiniband/hw/ehca/ehca_eq.c b/drivers/infiniband/hw/ehca/ehca_eq.c +index 49660df..aab80e9 100644 +--- a/drivers/infiniband/hw/ehca/ehca_eq.c ++++ b/drivers/infiniband/hw/ehca/ehca_eq.c +@@ -122,7 +122,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + /* register interrupt handlers and initialize work queues */ + if (type == EHCA_EQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_eq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_eq, + IRQF_DISABLED, "ehca_eq", + (void *)shca); + if (ret < 0) +@@ -130,7 +130,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + tasklet_init(&eq->interrupt_task, ehca_tasklet_eq, (long)shca); + } else if (type == EHCA_NEQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_neq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_neq, + IRQF_DISABLED, "ehca_neq", + (void *)shca); + if (ret < 0) +@@ -170,7 +170,7 @@ int ehca_destroy_eq(struct ehca_shca *shca, struct ehca_eq *eq) + u64 h_ret; + + spin_lock_irqsave(&eq->spinlock, flags); +- ibmebus_free_irq(eq->ist, (void *)shca); ++ ibmebus_free_irq(NULL, eq->ist, (void *)shca); + + h_ret = hipz_h_destroy_eq(shca->ipz_hca_handle, eq); + +diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c +index 482103e..666aafe 100644 +--- a/drivers/infiniband/hw/ehca/ehca_main.c ++++ b/drivers/infiniband/hw/ehca/ehca_main.c +@@ -289,8 +289,8 @@ static int ehca_sense_attributes(struct + }; + + ehca_gen_dbg("Probing adapter %s...", +- shca->ofdev->node->full_name); +- loc_code = of_get_property(shca->ofdev->node, "ibm,loc-code", NULL); ++ shca->ibmebus_dev->ofdev.node->full_name); ++ loc_code = of_get_property(shca->ibmebus_dev->ofdev.node, "ibm,loc-code", NULL); + if (loc_code) + ehca_gen_dbg(" ... location lode=%s", loc_code); + +@@ -458,7 +458,7 @@ static int ehca_init_device(struct ehca_shca *shca) + shca->ib_device.node_type = RDMA_NODE_IB_CA; + shca->ib_device.phys_port_cnt = shca->num_ports; + shca->ib_device.num_comp_vectors = 1; +- shca->ib_device.dma_device = &shca->ofdev->dev; ++ shca->ib_device.dma_device = &shca->ibmebus_dev->ofdev.dev; + shca->ib_device.query_device = ehca_query_device; + shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_gid = ehca_query_gid; +@@ -619,11 +619,6 @@ static struct attribute_group ehca_drv_attr_grp = { + .attrs = ehca_drv_attrs + }; + +-static struct attribute_group *ehca_drv_attr_groups[] = { +- &ehca_drv_attr_grp, +- NULL, +-}; +- + #define EHCA_RESOURCE_ATTR(name) \ + static ssize_t ehca_show_##name(struct device *dev, \ + struct device_attribute *attr, \ +@@ -707,7 +702,7 @@ static struct attribute_group ehca_dev_attr_grp = { + .attrs = ehca_dev_attrs + }; + +-static int __devinit ehca_probe(struct of_device *dev, ++static int __devinit ehca_probe(struct ibmebus_dev *dev, + const struct of_device_id *id) + { + struct ehca_shca *shca; +@@ -715,16 +710,16 @@ static int __devinit ehca_probe(struct of_device *dev, + struct ib_pd *ibpd; + int ret, i, eq_size; + +- handle = of_get_property(dev->node, "ibm,hca-handle", NULL); ++ handle = of_get_property(dev->ofdev.node, "ibm,hca-handle", NULL); + if (!handle) { + ehca_gen_err("Cannot get eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + + if (!(*handle)) { + ehca_gen_err("Wrong eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + +@@ -739,9 +734,9 @@ static int __devinit ehca_probe(struct of_device *dev, + for (i = 0; i < ARRAY_SIZE(shca->sport); i++) + spin_lock_init(&shca->sport[i].mod_sqp_lock); + +- shca->ofdev = dev; ++ shca->ibmebus_dev = dev; + shca->ipz_hca_handle.handle = *handle; +- dev->dev.driver_data = shca; ++ dev->ofdev.dev.driver_data = shca; + + ret = ehca_sense_attributes(shca); + if (ret < 0) { +@@ -818,7 +813,7 @@ static int __devinit ehca_probe(struct of_device *dev, + } + } + +- ret = sysfs_create_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ ret = sysfs_create_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + if (ret) /* only complain; we can live without attributes */ + ehca_err(&shca->ib_device, + "Cannot create device attributes ret=%d", ret); +@@ -868,12 +863,12 @@ probe1: + return -EINVAL; + } + +-static int __devexit ehca_remove(struct of_device *dev) ++static int __devexit ehca_remove(struct ibmebus_dev *dev) + { +- struct ehca_shca *shca = dev->dev.driver_data; ++ struct ehca_shca *shca = dev->ofdev.dev.driver_data; + int ret; + +- sysfs_remove_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ sysfs_remove_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + + if (ehca_open_aqp1 == 1) { + int i; +@@ -924,14 +919,11 @@ static struct of_device_id ehca_device_table[] = + {}, + }; + +-static struct of_platform_driver ehca_driver = { +- .name = "ehca", +- .match_table = ehca_device_table, +- .probe = ehca_probe, +- .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, ++static struct ibmebus_driver ehca_driver = { ++ .name = "ehca", ++ .id_table = ehca_device_table, ++ .probe = ehca_probe, ++ .remove = ehca_remove, + }; + + void ehca_poll_eqs(unsigned long data) +@@ -990,6 +982,10 @@ static int __init ehca_module_init(void) + goto module_init2; + } + ++ ret = sysfs_create_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); ++ if (ret) /* only complain; we can live without attributes */ ++ ehca_gen_err("Cannot create driver attributes ret=%d", ret); ++ + if (ehca_poll_all_eqs != 1) { + ehca_gen_err("WARNING!!!"); + ehca_gen_err("It is possible to lose interrupts."); +@@ -1015,6 +1011,7 @@ static void __exit ehca_module_exit(void) + if (ehca_poll_all_eqs == 1) + del_timer_sync(&poll_eqs_timer); + ++ sysfs_remove_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); + ibmebus_unregister_driver(&ehca_driver); + + ehca_destroy_slab_caches(); + diff --git a/kernel_patches/backport/2.6.18-EL5.2/ehca-01-ibmebus_loc_code.patch b/kernel_patches/backport/2.6.18-EL5.2/ehca-01-ibmebus_loc_code.patch new file mode 100644 index 0000000..8737b79 --- /dev/null +++ b/kernel_patches/backport/2.6.18-EL5.2/ehca-01-ibmebus_loc_code.patch @@ -0,0 +1,186 @@ +diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h +index 1e9e99a..fbf6e58 100644 +--- a/drivers/infiniband/hw/ehca/ehca_classes.h ++++ b/drivers/infiniband/hw/ehca/ehca_classes.h +@@ -112,7 +112,7 @@ struct ehca_sport { + + struct ehca_shca { + struct ib_device ib_device; +- struct of_device *ofdev; ++ struct ibmebus_dev *ibmebus_dev; + u8 num_ports; + int hw_level; + struct list_head shca_list; +diff --git a/drivers/infiniband/hw/ehca/ehca_eq.c b/drivers/infiniband/hw/ehca/ehca_eq.c +index 49660df..aab80e9 100644 +--- a/drivers/infiniband/hw/ehca/ehca_eq.c ++++ b/drivers/infiniband/hw/ehca/ehca_eq.c +@@ -122,7 +122,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + /* register interrupt handlers and initialize work queues */ + if (type == EHCA_EQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_eq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_eq, + IRQF_DISABLED, "ehca_eq", + (void *)shca); + if (ret < 0) +@@ -130,7 +130,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + tasklet_init(&eq->interrupt_task, ehca_tasklet_eq, (long)shca); + } else if (type == EHCA_NEQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_neq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_neq, + IRQF_DISABLED, "ehca_neq", + (void *)shca); + if (ret < 0) +@@ -170,7 +170,7 @@ int ehca_destroy_eq(struct ehca_shca *shca, struct ehca_eq *eq) + u64 h_ret; + + spin_lock_irqsave(&eq->spinlock, flags); +- ibmebus_free_irq(eq->ist, (void *)shca); ++ ibmebus_free_irq(NULL, eq->ist, (void *)shca); + + h_ret = hipz_h_destroy_eq(shca->ipz_hca_handle, eq); + +diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c +index 482103e..666aafe 100644 +--- a/drivers/infiniband/hw/ehca/ehca_main.c ++++ b/drivers/infiniband/hw/ehca/ehca_main.c +@@ -289,8 +289,8 @@ static int ehca_sense_attributes(struct + }; + + ehca_gen_dbg("Probing adapter %s...", +- shca->ofdev->node->full_name); +- loc_code = of_get_property(shca->ofdev->node, "ibm,loc-code", NULL); ++ shca->ibmebus_dev->ofdev.node->full_name); ++ loc_code = of_get_property(shca->ibmebus_dev->ofdev.node, "ibm,loc-code", NULL); + if (loc_code) + ehca_gen_dbg(" ... location lode=%s", loc_code); + +@@ -458,7 +458,7 @@ static int ehca_init_device(struct ehca_shca *shca) + shca->ib_device.node_type = RDMA_NODE_IB_CA; + shca->ib_device.phys_port_cnt = shca->num_ports; + shca->ib_device.num_comp_vectors = 1; +- shca->ib_device.dma_device = &shca->ofdev->dev; ++ shca->ib_device.dma_device = &shca->ibmebus_dev->ofdev.dev; + shca->ib_device.query_device = ehca_query_device; + shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_gid = ehca_query_gid; +@@ -619,11 +619,6 @@ static struct attribute_group ehca_drv_attr_grp = { + .attrs = ehca_drv_attrs + }; + +-static struct attribute_group *ehca_drv_attr_groups[] = { +- &ehca_drv_attr_grp, +- NULL, +-}; +- + #define EHCA_RESOURCE_ATTR(name) \ + static ssize_t ehca_show_##name(struct device *dev, \ + struct device_attribute *attr, \ +@@ -707,7 +702,7 @@ static struct attribute_group ehca_dev_attr_grp = { + .attrs = ehca_dev_attrs + }; + +-static int __devinit ehca_probe(struct of_device *dev, ++static int __devinit ehca_probe(struct ibmebus_dev *dev, + const struct of_device_id *id) + { + struct ehca_shca *shca; +@@ -715,16 +710,16 @@ static int __devinit ehca_probe(struct of_device *dev, + struct ib_pd *ibpd; + int ret, i, eq_size; + +- handle = of_get_property(dev->node, "ibm,hca-handle", NULL); ++ handle = of_get_property(dev->ofdev.node, "ibm,hca-handle", NULL); + if (!handle) { + ehca_gen_err("Cannot get eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + + if (!(*handle)) { + ehca_gen_err("Wrong eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + +@@ -739,9 +734,9 @@ static int __devinit ehca_probe(struct of_device *dev, + for (i = 0; i < ARRAY_SIZE(shca->sport); i++) + spin_lock_init(&shca->sport[i].mod_sqp_lock); + +- shca->ofdev = dev; ++ shca->ibmebus_dev = dev; + shca->ipz_hca_handle.handle = *handle; +- dev->dev.driver_data = shca; ++ dev->ofdev.dev.driver_data = shca; + + ret = ehca_sense_attributes(shca); + if (ret < 0) { +@@ -818,7 +813,7 @@ static int __devinit ehca_probe(struct of_device *dev, + } + } + +- ret = sysfs_create_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ ret = sysfs_create_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + if (ret) /* only complain; we can live without attributes */ + ehca_err(&shca->ib_device, + "Cannot create device attributes ret=%d", ret); +@@ -868,12 +863,12 @@ probe1: + return -EINVAL; + } + +-static int __devexit ehca_remove(struct of_device *dev) ++static int __devexit ehca_remove(struct ibmebus_dev *dev) + { +- struct ehca_shca *shca = dev->dev.driver_data; ++ struct ehca_shca *shca = dev->ofdev.dev.driver_data; + int ret; + +- sysfs_remove_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ sysfs_remove_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + + if (ehca_open_aqp1 == 1) { + int i; +@@ -924,14 +919,11 @@ static struct of_device_id ehca_device_table[] = + {}, + }; + +-static struct of_platform_driver ehca_driver = { +- .name = "ehca", +- .match_table = ehca_device_table, +- .probe = ehca_probe, +- .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, ++static struct ibmebus_driver ehca_driver = { ++ .name = "ehca", ++ .id_table = ehca_device_table, ++ .probe = ehca_probe, ++ .remove = ehca_remove, + }; + + void ehca_poll_eqs(unsigned long data) +@@ -990,6 +982,10 @@ static int __init ehca_module_init(void) + goto module_init2; + } + ++ ret = sysfs_create_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); ++ if (ret) /* only complain; we can live without attributes */ ++ ehca_gen_err("Cannot create driver attributes ret=%d", ret); ++ + if (ehca_poll_all_eqs != 1) { + ehca_gen_err("WARNING!!!"); + ehca_gen_err("It is possible to lose interrupts."); +@@ -1015,6 +1011,7 @@ static void __exit ehca_module_exit(void) + if (ehca_poll_all_eqs == 1) + del_timer_sync(&poll_eqs_timer); + ++ sysfs_remove_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); + ibmebus_unregister_driver(&ehca_driver); + + ehca_destroy_slab_caches(); + -- 1.5.5 From fenkes at de.ibm.com Mon Jul 14 12:33:56 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 14 Jul 2008 21:33:56 +0200 Subject: [ofa-general] [PATCH 5/5] IB/ehca: SLES10 SP1/2 backports In-Reply-To: <200807142128.53998.fenkes@de.ibm.com> References: <200807142128.53998.fenkes@de.ibm.com> Message-ID: <200807142133.57108.fenkes@de.ibm.com> From: Alexander Schmidt Signed-off-by: Joachim Fenkes --- .../ehca_01_ibmebus_loc_code.patch | 154 ++++++++++++++++++++ .../ehca_01_ibmebus_loc_code.patch | 154 ++++++++++++++++++++ 2 files changed, 308 insertions(+), 0 deletions(-) create mode 100644 kernel_patches/backport/2.6.16_sles10_sp1/ehca_01_ibmebus_loc_code.patch create mode 100644 kernel_patches/backport/2.6.16_sles10_sp2/ehca_01_ibmebus_loc_code.patch diff --git a/kernel_patches/backport/2.6.16_sles10_sp1/ehca_01_ibmebus_loc_code.patch b/kernel_patches/backport/2.6.16_sles10_sp1/ehca_01_ibmebus_loc_code.patch new file mode 100644 index 0000000..9d4e606 --- /dev/null +++ b/kernel_patches/backport/2.6.16_sles10_sp1/ehca_01_ibmebus_loc_code.patch @@ -0,0 +1,154 @@ +Index: ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_classes.h +=================================================================== +--- ofa_kernel-1.4.orig/drivers/infiniband/hw/ehca/ehca_classes.h 2008-07-07 15:06:10.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_classes.h 2008-07-10 01:33:03.000000000 +0200 +@@ -112,7 +112,7 @@ + + struct ehca_shca { + struct ib_device ib_device; +- struct of_device *ofdev; ++ struct ibmebus_dev *ibmebus_dev; + u8 num_ports; + int hw_level; + struct list_head shca_list; +Index: ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_eq.c +=================================================================== +--- ofa_kernel-1.4.orig/drivers/infiniband/hw/ehca/ehca_eq.c 2008-07-07 15:06:10.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_eq.c 2008-07-10 01:33:03.000000000 +0200 +@@ -122,7 +122,7 @@ + + /* register interrupt handlers and initialize work queues */ + if (type == EHCA_EQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_eq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_eq, + IRQF_DISABLED, "ehca_eq", + (void *)shca); + if (ret < 0) +@@ -130,7 +130,7 @@ + + tasklet_init(&eq->interrupt_task, ehca_tasklet_eq, (long)shca); + } else if (type == EHCA_NEQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_neq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_neq, + IRQF_DISABLED, "ehca_neq", + (void *)shca); + if (ret < 0) +@@ -170,7 +170,7 @@ + u64 h_ret; + + spin_lock_irqsave(&eq->spinlock, flags); +- ibmebus_free_irq(eq->ist, (void *)shca); ++ ibmebus_free_irq(NULL, eq->ist, (void *)shca); + + h_ret = hipz_h_destroy_eq(shca->ipz_hca_handle, eq); + +Index: ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_main.c +=================================================================== +--- ofa_kernel-1.4.orig/drivers/infiniband/hw/ehca/ehca_main.c 2008-07-10 01:32:25.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_main.c 2008-07-10 01:35:21.000000000 +0200 +@@ -289,8 +289,8 @@ + }; + + ehca_gen_dbg("Probing adapter %s...", +- shca->ofdev->node->full_name); +- loc_code = of_get_property(shca->ofdev->node, "ibm,loc-code", NULL); ++ shca->ibmebus_dev->ofdev.node->full_name); ++ loc_code = of_get_property(shca->ibmebus_dev->ofdev.node, "ibm,loc-code", NULL); + if (loc_code) + ehca_gen_dbg(" ... location lode=%s", loc_code); + +@@ -458,7 +458,7 @@ + shca->ib_device.node_type = RDMA_NODE_IB_CA; + shca->ib_device.phys_port_cnt = shca->num_ports; + shca->ib_device.num_comp_vectors = 1; +- shca->ib_device.dma_device = &shca->ofdev->dev; ++ shca->ib_device.dma_device = &shca->ibmebus_dev->ofdev.dev; + shca->ib_device.query_device = ehca_query_device; + shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_gid = ehca_query_gid; +@@ -707,7 +707,7 @@ + .attrs = ehca_dev_attrs + }; + +-static int __devinit ehca_probe(struct of_device *dev, ++static int __devinit ehca_probe(struct ibmebus_dev *dev, + const struct of_device_id *id) + { + struct ehca_shca *shca; +@@ -715,16 +715,16 @@ + struct ib_pd *ibpd; + int ret, i, eq_size; + +- handle = of_get_property(dev->node, "ibm,hca-handle", NULL); ++ handle = of_get_property(dev->ofdev.node, "ibm,hca-handle", NULL); + if (!handle) { + ehca_gen_err("Cannot get eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + + if (!(*handle)) { + ehca_gen_err("Wrong eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + +@@ -739,9 +739,9 @@ + for (i = 0; i < ARRAY_SIZE(shca->sport); i++) + spin_lock_init(&shca->sport[i].mod_sqp_lock); + +- shca->ofdev = dev; ++ shca->ibmebus_dev = dev; + shca->ipz_hca_handle.handle = *handle; +- dev->dev.driver_data = shca; ++ dev->ofdev.dev.driver_data = shca; + + ret = ehca_sense_attributes(shca); + if (ret < 0) { +@@ -818,7 +818,7 @@ + } + } + +- ret = sysfs_create_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ ret = sysfs_create_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + if (ret) /* only complain; we can live without attributes */ + ehca_err(&shca->ib_device, + "Cannot create device attributes ret=%d", ret); +@@ -868,12 +868,12 @@ + return -EINVAL; + } + +-static int __devexit ehca_remove(struct of_device *dev) ++static int __devexit ehca_remove(struct ibmebus_dev *dev) + { +- struct ehca_shca *shca = dev->dev.driver_data; ++ struct ehca_shca *shca = dev->ofdev.dev.driver_data; + int ret; + +- sysfs_remove_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ sysfs_remove_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + + if (ehca_open_aqp1 == 1) { + int i; +@@ -924,14 +924,11 @@ + {}, + }; + +-static struct of_platform_driver ehca_driver = { +- .name = "ehca", +- .match_table = ehca_device_table, +- .probe = ehca_probe, +- .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, ++static struct ibmebus_driver ehca_driver = { ++ .name = "ehca", ++ .id_table = ehca_device_table, ++ .probe = ehca_probe, ++ .remove = ehca_remove, + }; + + void ehca_poll_eqs(unsigned long data) diff --git a/kernel_patches/backport/2.6.16_sles10_sp2/ehca_01_ibmebus_loc_code.patch b/kernel_patches/backport/2.6.16_sles10_sp2/ehca_01_ibmebus_loc_code.patch new file mode 100644 index 0000000..9d4e606 --- /dev/null +++ b/kernel_patches/backport/2.6.16_sles10_sp2/ehca_01_ibmebus_loc_code.patch @@ -0,0 +1,154 @@ +Index: ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_classes.h +=================================================================== +--- ofa_kernel-1.4.orig/drivers/infiniband/hw/ehca/ehca_classes.h 2008-07-07 15:06:10.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_classes.h 2008-07-10 01:33:03.000000000 +0200 +@@ -112,7 +112,7 @@ + + struct ehca_shca { + struct ib_device ib_device; +- struct of_device *ofdev; ++ struct ibmebus_dev *ibmebus_dev; + u8 num_ports; + int hw_level; + struct list_head shca_list; +Index: ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_eq.c +=================================================================== +--- ofa_kernel-1.4.orig/drivers/infiniband/hw/ehca/ehca_eq.c 2008-07-07 15:06:10.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_eq.c 2008-07-10 01:33:03.000000000 +0200 +@@ -122,7 +122,7 @@ + + /* register interrupt handlers and initialize work queues */ + if (type == EHCA_EQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_eq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_eq, + IRQF_DISABLED, "ehca_eq", + (void *)shca); + if (ret < 0) +@@ -130,7 +130,7 @@ + + tasklet_init(&eq->interrupt_task, ehca_tasklet_eq, (long)shca); + } else if (type == EHCA_NEQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_neq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_neq, + IRQF_DISABLED, "ehca_neq", + (void *)shca); + if (ret < 0) +@@ -170,7 +170,7 @@ + u64 h_ret; + + spin_lock_irqsave(&eq->spinlock, flags); +- ibmebus_free_irq(eq->ist, (void *)shca); ++ ibmebus_free_irq(NULL, eq->ist, (void *)shca); + + h_ret = hipz_h_destroy_eq(shca->ipz_hca_handle, eq); + +Index: ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_main.c +=================================================================== +--- ofa_kernel-1.4.orig/drivers/infiniband/hw/ehca/ehca_main.c 2008-07-10 01:32:25.000000000 +0200 ++++ ofa_kernel-1.4/drivers/infiniband/hw/ehca/ehca_main.c 2008-07-10 01:35:21.000000000 +0200 +@@ -289,8 +289,8 @@ + }; + + ehca_gen_dbg("Probing adapter %s...", +- shca->ofdev->node->full_name); +- loc_code = of_get_property(shca->ofdev->node, "ibm,loc-code", NULL); ++ shca->ibmebus_dev->ofdev.node->full_name); ++ loc_code = of_get_property(shca->ibmebus_dev->ofdev.node, "ibm,loc-code", NULL); + if (loc_code) + ehca_gen_dbg(" ... location lode=%s", loc_code); + +@@ -458,7 +458,7 @@ + shca->ib_device.node_type = RDMA_NODE_IB_CA; + shca->ib_device.phys_port_cnt = shca->num_ports; + shca->ib_device.num_comp_vectors = 1; +- shca->ib_device.dma_device = &shca->ofdev->dev; ++ shca->ib_device.dma_device = &shca->ibmebus_dev->ofdev.dev; + shca->ib_device.query_device = ehca_query_device; + shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_gid = ehca_query_gid; +@@ -707,7 +707,7 @@ + .attrs = ehca_dev_attrs + }; + +-static int __devinit ehca_probe(struct of_device *dev, ++static int __devinit ehca_probe(struct ibmebus_dev *dev, + const struct of_device_id *id) + { + struct ehca_shca *shca; +@@ -715,16 +715,16 @@ + struct ib_pd *ibpd; + int ret, i, eq_size; + +- handle = of_get_property(dev->node, "ibm,hca-handle", NULL); ++ handle = of_get_property(dev->ofdev.node, "ibm,hca-handle", NULL); + if (!handle) { + ehca_gen_err("Cannot get eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + + if (!(*handle)) { + ehca_gen_err("Wrong eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + +@@ -739,9 +739,9 @@ + for (i = 0; i < ARRAY_SIZE(shca->sport); i++) + spin_lock_init(&shca->sport[i].mod_sqp_lock); + +- shca->ofdev = dev; ++ shca->ibmebus_dev = dev; + shca->ipz_hca_handle.handle = *handle; +- dev->dev.driver_data = shca; ++ dev->ofdev.dev.driver_data = shca; + + ret = ehca_sense_attributes(shca); + if (ret < 0) { +@@ -818,7 +818,7 @@ + } + } + +- ret = sysfs_create_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ ret = sysfs_create_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + if (ret) /* only complain; we can live without attributes */ + ehca_err(&shca->ib_device, + "Cannot create device attributes ret=%d", ret); +@@ -868,12 +868,12 @@ + return -EINVAL; + } + +-static int __devexit ehca_remove(struct of_device *dev) ++static int __devexit ehca_remove(struct ibmebus_dev *dev) + { +- struct ehca_shca *shca = dev->dev.driver_data; ++ struct ehca_shca *shca = dev->ofdev.dev.driver_data; + int ret; + +- sysfs_remove_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ sysfs_remove_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + + if (ehca_open_aqp1 == 1) { + int i; +@@ -924,14 +924,11 @@ + {}, + }; + +-static struct of_platform_driver ehca_driver = { +- .name = "ehca", +- .match_table = ehca_device_table, +- .probe = ehca_probe, +- .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, ++static struct ibmebus_driver ehca_driver = { ++ .name = "ehca", ++ .id_table = ehca_device_table, ++ .probe = ehca_probe, ++ .remove = ehca_remove, + }; + + void ehca_poll_eqs(unsigned long data) -- 1.5.5 From richard.frank at oracle.com Mon Jul 14 22:33:05 2008 From: richard.frank at oracle.com (Richard Frank) Date: Tue, 15 Jul 2008 01:33:05 -0400 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support References: <20080703213412.GB32285@opengridcomputing.com><200807062138.29103.olaf.kirch@oracle.com><48733D5C.7060000@voltaire.com> <200807081400.17025.olaf.kirch@oracle.com> Message-ID: <005b01c8e63c$4492d980$3edc10ac@YOURA06808D9DE> This all seems to be a bit restrictive - which will require some changes in how Oracle uses the IPC for rdma operations - but we need to start somewhere so this seems reasonable. If we can get bcopy running with Oracle - that'd be a great goal for 1.4. If it's going to require changes in Oracle to use IWARP RDMA - then we're talking about a later Orace release to use it... I'm out on vacation will be back in two weeks.. Rick ----- Original Message ----- From: "Olaf Kirch" To: "Or Gerlitz" Cc: ; Sent: Tuesday, July 08, 2008 8:00 AM Subject: Re: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support > On Tuesday 08 July 2008 12:11:40 Or Gerlitz wrote: >> Olaf Kirch wrote: >> > Okay, there's a third way to fix this, which may even work with today's >> > Oracle IPC code. The hack is to not submit the fastreg WR until we post >> > a SEND that transmits the rkey to a peer. Then >> > >> > - we have a QP to post the WRs to >> > - we post the fastreg and the SEND to the same QP, >> > ensuring that the peer doesn't initiate the RDMA >> > before we've set up the memory registration. >> Yes, as Steve noted, using the same QP for the fastreg and send that >> carries the rkey/stag is a must since the send should be fenced on the >> fastreg completion to make sure the remote side uses a valid key. > > One theoretical problem is what happens if the application wants to > register a memory range and send the rkey to two peers at the same time? > We would have to stall the second send explicitly as long as we > haven't seen the completion for the first send. > > Olaf > -- > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From dotanba at gmail.com Mon Jul 14 23:39:47 2008 From: dotanba at gmail.com (Dotan Barak) Date: Tue, 15 Jul 2008 09:39:47 +0300 Subject: [ofa-general] IB verbs client/server programming... In-Reply-To: <1216054280.15841.28.camel@hermosa.morrealenet> References: <1216054280.15841.28.camel@hermosa.morrealenet> Message-ID: <2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com> Hi. On Mon, Jul 14, 2008 at 7:51 PM, Peter W. Morreale wrote: > > Hello all, > > I'm starting a new client/server project using RDMA and I'm a little > stuck at the moment. > > I've seen Roland Dreier's slides/client-server example as well as the > cmatose.c example in the OFED 1.3 tree. Conceptually, I'm starting to > become familiar with channels and PD's and MR's ("Oh my!"), however one > thing is not clear to me at all. How do I block waiting for the next > RDMA event in my daemon? > > In other words, what interfaces do I have to look at to implement the > "blocking" portion equivalent of a recv(2)? Do you want to block until the data will be received? (if so: what is the meaning of this when the sender uses RDMA?) What exactly to you want to do? > > Seems like all the server examples block at rdma_listen(), then run to > completion exchanging data. > > Also, is there any documentation out there on using ib verbs? I looked > on the OpenFabrics web site (as well as Googled) and have found little > besides the man pages. Unfortunately, there isn't much documentation on this today. > > Thanks in advance for any pointers. > > -PWM > Dotan From jackm at dev.mellanox.co.il Mon Jul 14 23:45:34 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 15 Jul 2008 09:45:34 +0300 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: References: Message-ID: <200807150945.34571.jackm@dev.mellanox.co.il> On Monday 14 July 2008 08:16, Roland Dreier wrote: > HW specific: > >  - Yevgeny's mlx4 changes.  We'll see how much time is left after I >    get done with XRC (which is before this on my list) but to be >    honest I'm not sure how mergeable a lot of this is without the >    mlx4_en patches that actually use it. > >  - I've been working on memory management extensions support for mlx4, >    but I'm not sure if it will be ready in time.  Firmware for this >    may not be released for a while so it ain't urgent anyway. > Roland, what about the fw diagnostic patch for the mlx4 driver? http://lists.openfabrics.org/pipermail/general/2008-June/051655.html Can you put this in 2.6.27, too? - Jack From rdreier at cisco.com Mon Jul 14 23:51:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 14 Jul 2008 23:51:11 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get the first big batch of RDMA/InfiniBand changes for 2.6.27, all of which have been in linux-next: Christophe Jaillet (1): RDMA/nes: Remove unnecessary memset() Dotan Barak (1): RDMA: Improve include file coding style Eli Cohen (12): IB/mlx4: Optimize QP stamping IPoIB: Copy small received SKBs in connected mode IB/mlx4: Configure QPs' max message size based on real device capability IB/mlx4: Pass congestion management class MADs to the HCA IPoIB: Remove unused IPOIB_MCAST_STARTED code IPoIB: Remove priv->mcast_mutex IPoIB: Only set Q_Key once: after joining broadcast group IPoIB: Use rtnl lock/unlock when changing device flags IPoIB: Use dev_set_mtu() to change mtu IPoIB/cm: Reduce connected mode TX object size IPoIB: Double default RX/TX ring sizes IB/mlx4: Use kzalloc() for new QPs so flags are initialized to 0 Joachim Fenkes (2): IB/ehca: Reject receive work requests if QP is in RESET state IB/ehca: Make device table externally visible Jon Mason (1): RDMA/cxgb3: Propagate HW page size capabilities Moni Shoua (2): IB/sa: Fail requests made while creating new SM AH IPoIB: Refresh paths instead of flushing them on SM change events Or Gerlitz (2): RDMA/addr: Keep pointer to netdevice in struct rdma_dev_addr RDMA/cma: Simplify locking needed for serialization of callbacks Ralph Campbell (2): IB/core: Reset to error QP state transition is not allowed IB/ipath: Use IEEE OUI for vendor_id reported by ibv_query_device() Robert P. J. Day (1): IB/ipath: Simplify code using ARRAY_SIZE() macro Roland Dreier (13): IB/srp: Remove use of cached P_Key/GID queries RDMA: Remove subversion $Id tags IB/mthca: Remove extra code for RESET->ERR QP state transition IB/mlx4: Remove extra code for RESET->ERR QP state transition RDMA/cxgb3: Remove write-only iwch_rnic_attributes fields RDMA/cma: Add missing newlines to printk()s IPoIB/cm: Fix racy use of receive WR/SGL in ipoib_cm_post_receive_nonsrq() RDMA/nes: Encapsulate logic nes_put_cqp_request() RDMA/nes: Get rid of ring_doorbell parameter of nes_post_cqp_request() IPoIB: Get rid of ipoib_mcast_detach() wrapper IB/mthca: Remove "stop" flag for catastrophic error polling timer IB/mthca: Use round_jiffies() for catastrophic error polling timer IB/mthca: Fix check of max_send_sge for special QPs Ron Livne (3): IB/core: Add support for multicast loopback blocking IB/mlx4: Add support for blocking multicast loopback packets IPoIB: Use multicast loopback blocking if available Sean Hefty (1): RDMA: Fix license text Stefan Roscher (1): IB/ehca: In case of lost interrupts, trigger EOI to reenable interrupts Steve Wise (8): RDMA/core: Add memory management extensions support RDMA/cxgb3: MEM_MGT_EXTENSIONS support RDMA/cxgb3: Fix up some ib_device_attr fields RDMA/core: Add iWARP protocol statistics attributes in sysfs RDMA/cxgb3: Add support for protocol statistics RDMA/cxgb3: Set rkey field for new memory windows in iwch_alloc_mw() RDMA/core: Add local DMA L_Key support RDMA/cxgb3: Fixes for zero STag Vladimir Sokolovsky (2): IPoIB: add LRO support mlx4_core: Use MOD_STAT_CFG command to get minimal page size drivers/infiniband/core/addr.c | 42 +++-- drivers/infiniband/core/agent.h | 2 - drivers/infiniband/core/cache.c | 2 - drivers/infiniband/core/cm.c | 2 - drivers/infiniband/core/cma.c | 162 ++++++++-------- drivers/infiniband/core/core_priv.h | 2 - drivers/infiniband/core/device.c | 2 - drivers/infiniband/core/fmr_pool.c | 2 - drivers/infiniband/core/mad_priv.h | 2 - drivers/infiniband/core/mad_rmpp.c | 2 - drivers/infiniband/core/mad_rmpp.h | 2 - drivers/infiniband/core/packer.c | 2 - drivers/infiniband/core/sa_query.c | 24 ++- drivers/infiniband/core/sysfs.c | 122 +++++++++++- drivers/infiniband/core/ucm.c | 2 - drivers/infiniband/core/ud_header.c | 2 - drivers/infiniband/core/umem.c | 2 - drivers/infiniband/core/user_mad.c | 2 - drivers/infiniband/core/uverbs.h | 2 - drivers/infiniband/core/uverbs_cmd.c | 4 +- drivers/infiniband/core/uverbs_main.c | 2 - drivers/infiniband/core/verbs.c | 49 ++++- drivers/infiniband/hw/amso1100/c2_rnic.c | 2 +- drivers/infiniband/hw/cxgb3/cxio_hal.c | 27 ++- drivers/infiniband/hw/cxgb3/cxio_hal.h | 5 +- drivers/infiniband/hw/cxgb3/cxio_wr.h | 103 ++++++++-- drivers/infiniband/hw/cxgb3/iwch.c | 8 +- drivers/infiniband/hw/cxgb3/iwch.h | 2 - drivers/infiniband/hw/cxgb3/iwch_cq.c | 15 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 199 +++++++++++++++++- drivers/infiniband/hw/cxgb3/iwch_provider.h | 8 - drivers/infiniband/hw/cxgb3/iwch_qp.c | 261 ++++++++++++++++++------ drivers/infiniband/hw/ehca/ehca_irq.c | 9 +- drivers/infiniband/hw/ehca/ehca_main.c | 1 + drivers/infiniband/hw/ehca/ehca_reqs.c | 14 +- drivers/infiniband/hw/ehca/hcp_if.c | 10 + drivers/infiniband/hw/ehca/hcp_if.h | 1 + drivers/infiniband/hw/ipath/ipath_cq.c | 2 +- drivers/infiniband/hw/ipath/ipath_iba7220.c | 4 +- drivers/infiniband/hw/ipath/ipath_mad.c | 6 +- drivers/infiniband/hw/ipath/ipath_rc.c | 4 +- drivers/infiniband/hw/ipath/ipath_ruc.c | 4 +- drivers/infiniband/hw/ipath/ipath_uc.c | 8 +- drivers/infiniband/hw/ipath/ipath_ud.c | 8 +- drivers/infiniband/hw/ipath/ipath_verbs.c | 3 +- drivers/infiniband/hw/mlx4/cq.c | 12 +- drivers/infiniband/hw/mlx4/mad.c | 3 +- drivers/infiniband/hw/mlx4/main.c | 7 +- drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 +- drivers/infiniband/hw/mlx4/qp.c | 73 +++---- drivers/infiniband/hw/mthca/mthca_allocator.c | 2 - drivers/infiniband/hw/mthca/mthca_av.c | 2 - drivers/infiniband/hw/mthca/mthca_catas.c | 17 +-- drivers/infiniband/hw/mthca/mthca_cmd.c | 2 - drivers/infiniband/hw/mthca/mthca_cmd.h | 2 - drivers/infiniband/hw/mthca/mthca_config_reg.h | 2 - drivers/infiniband/hw/mthca/mthca_cq.c | 6 +- drivers/infiniband/hw/mthca/mthca_dev.h | 3 - drivers/infiniband/hw/mthca/mthca_doorbell.h | 2 - drivers/infiniband/hw/mthca/mthca_eq.c | 2 - drivers/infiniband/hw/mthca/mthca_mad.c | 2 - drivers/infiniband/hw/mthca/mthca_main.c | 2 - drivers/infiniband/hw/mthca/mthca_mcg.c | 2 - drivers/infiniband/hw/mthca/mthca_memfree.c | 2 - drivers/infiniband/hw/mthca/mthca_memfree.h | 2 - drivers/infiniband/hw/mthca/mthca_mr.c | 2 - drivers/infiniband/hw/mthca/mthca_pd.c | 2 - drivers/infiniband/hw/mthca/mthca_profile.c | 2 - drivers/infiniband/hw/mthca/mthca_profile.h | 2 - drivers/infiniband/hw/mthca/mthca_provider.c | 2 - drivers/infiniband/hw/mthca/mthca_provider.h | 2 - drivers/infiniband/hw/mthca/mthca_qp.c | 32 +--- drivers/infiniband/hw/mthca/mthca_reset.c | 2 - drivers/infiniband/hw/mthca/mthca_srq.c | 2 - drivers/infiniband/hw/mthca/mthca_uar.c | 2 - drivers/infiniband/hw/mthca/mthca_user.h | 1 - drivers/infiniband/hw/mthca/mthca_wqe.h | 2 - drivers/infiniband/hw/nes/nes.c | 2 +- drivers/infiniband/hw/nes/nes.h | 9 +- drivers/infiniband/hw/nes/nes_cm.c | 1 - drivers/infiniband/hw/nes/nes_hw.c | 68 +----- drivers/infiniband/hw/nes/nes_hw.h | 2 +- drivers/infiniband/hw/nes/nes_utils.c | 33 +++- drivers/infiniband/hw/nes/nes_verbs.c | 207 ++++--------------- drivers/infiniband/ulp/ipoib/Kconfig | 1 + drivers/infiniband/ulp/ipoib/ipoib.h | 48 ++++-- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 104 +++++++--- drivers/infiniband/ulp/ipoib/ipoib_ethtool.c | 46 ++++ drivers/infiniband/ulp/ipoib/ipoib_fs.c | 2 - drivers/infiniband/ulp/ipoib/ipoib_ib.c | 52 +++-- drivers/infiniband/ulp/ipoib/ipoib_main.c | 115 ++++++++++- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 27 +-- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 69 +++---- drivers/infiniband/ulp/ipoib/ipoib_vlan.c | 2 - drivers/infiniband/ulp/iser/iscsi_iser.c | 3 - drivers/infiniband/ulp/iser/iscsi_iser.h | 2 - drivers/infiniband/ulp/iser/iser_initiator.c | 2 - drivers/infiniband/ulp/iser/iser_memory.c | 2 - drivers/infiniband/ulp/iser/iser_verbs.c | 2 - drivers/infiniband/ulp/srp/ib_srp.c | 15 +- drivers/infiniband/ulp/srp/ib_srp.h | 2 - drivers/net/cxgb3/cxgb3_ctl_defs.h | 1 + drivers/net/cxgb3/cxgb3_offload.c | 7 + drivers/net/cxgb3/version.h | 2 +- drivers/net/mlx4/fw.c | 28 +++ drivers/net/mlx4/fw.h | 6 + drivers/net/mlx4/main.c | 7 + drivers/net/mlx4/mcg.c | 17 ++- include/linux/mlx4/device.h | 3 +- include/rdma/ib_addr.h | 43 +++-- include/rdma/ib_cache.h | 2 - include/rdma/ib_cm.h | 2 - include/rdma/ib_fmr_pool.h | 4 +- include/rdma/ib_mad.h | 17 +- include/rdma/ib_pack.h | 2 - include/rdma/ib_sa.h | 2 - include/rdma/ib_smi.h | 4 +- include/rdma/ib_user_cm.h | 2 - include/rdma/ib_user_mad.h | 2 - include/rdma/ib_user_verbs.h | 7 +- include/rdma/ib_verbs.h | 149 +++++++++++++- include/rdma/iw_cm.h | 2 +- include/rdma/rdma_cm.h | 52 +++--- include/rdma/rdma_cm_ib.h | 50 +++-- 124 files changed, 1632 insertions(+), 935 deletions(-) From rdreier at cisco.com Mon Jul 14 23:52:15 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 14 Jul 2008 23:52:15 -0700 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: <20080714135009.GA29500@mtls03> (Eli Cohen's message of "Mon, 14 Jul 2008 16:50:09 +0300") References: <20080714135009.GA29500@mtls03> Message-ID: > There is this patch that I did not recieve your response for. We think > it's reasonable to do this - what do you think? > > http://lists.openfabrics.org/pipermail/general/2008-July/052460.html I'm ambivalent. Seems like a minor usability improvement in some cases, but on the other hand it seems a little strange to change the MTU behind the user's back. Maybe I'll stick it in -- I need to think about it. From rdreier at cisco.com Mon Jul 14 23:52:30 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 14 Jul 2008 23:52:30 -0700 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: <200807150945.34571.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Tue, 15 Jul 2008 09:45:34 +0300") References: <200807150945.34571.jackm@dev.mellanox.co.il> Message-ID: > Roland, what about the fw diagnostic patch for the mlx4 driver? > http://lists.openfabrics.org/pipermail/general/2008-June/051655.html Oh yeah, I'll reply on that thread. From anuj01 at gmail.com Mon Jul 14 23:54:14 2008 From: anuj01 at gmail.com (=?UTF-8?B?4KSF4KSo4KWB4KSc?=) Date: Tue, 15 Jul 2008 12:24:14 +0530 Subject: [ofa-general] Hi - Uverbs flow path Message-ID: Hi I am studying the details of uverbs working. For this I want to put my code in the source file of some libraries like in libmthca (libmthca/src/mthca.c). I hav already downloaded the .tar file of libmthca and made changes to mthca.c. How can i rebuild this library and install it? And there is no autgen.sh script in the downloaded libmthca. regards, Anuj Aggarwal From rdreier at cisco.com Mon Jul 14 23:57:39 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 14 Jul 2008 23:57:39 -0700 Subject: [ofa-general] Re: [PATCH v2] mlx4: make firmware diagnostic counters available via sysfs In-Reply-To: <200807090947.29539.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Wed, 9 Jul 2008 09:47:29 +0300") References: <200804021615.44982.jackm@dev.mellanox.co.il> <200806181232.35634.jackm@dev.mellanox.co.il> <200807090947.29539.jackm@dev.mellanox.co.il> Message-ID: > The firmware diagnostic counters are not IB-protocol counters -- they are low-level > device-specific (i.e., ConnectX-specific) counters. Hmm, I would disagree... things like "Responder - number of local length errors" seem pretty close to, say, InTooBigErrors in the IP MIB. > If we are to implement exposing these firmware counters in the core layer, we need > a low-level driver API which would allow the low-level driver to pass to the core > layer at run time a vector of counter names, and we would then need to do all the > compile-time operations (DEVICE_ATTR, declaration of all the "show functions", etc) > at run time. > It looks to me that this would be messy. For example, each "show" method is specific to > one counter-name "file". We would then need to pre-declare (at compile time) a group > of "show" methods whose names are the offsets, rather than the counter-name > (e.g., device_name ## "counter-1", etc.), and this would have the effect of limiting > the max number of counters we could generate. this would be not too hard to implement in the same way as "ethtool -S" does it I guess. but I'm not keen on sticking a bunch of cryptic hardware-device-specific junk in sysfs. - R. From rdreier at cisco.com Mon Jul 14 23:59:06 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 14 Jul 2008 23:59:06 -0700 Subject: [ofa-general] Hi - Uverbs flow path In-Reply-To: (=?utf-8?B?IuCkheCkqOClgeCknCIncw==?= message of "Tue, 15 Jul 2008 12:24:14 +0530") References: Message-ID: > I hav already downloaded the .tar file of libmthca and made changes to mthca.c. > > How can i rebuild this library and install it? ./configure && make install > And there is no autgen.sh script in the downloaded libmthca. Right, because all the autotools have already been run. From jackm at dev.mellanox.co.il Tue Jul 15 00:00:58 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 15 Jul 2008 10:00:58 +0300 Subject: [ofa-general] [PATCH] mlx4: Change release date to 2008, and version to 1.0 (since is GA) - for 2.6.27 Message-ID: <200807151000.59190.jackm@dev.mellanox.co.il> From 63e055fcefc78c919b654ac2c4b3db354c44c226 Mon Sep 17 00:00:00 2001 From: root Date: Tue, 15 Jul 2008 09:58:32 +0300 Subject: [PATCH] mlx4: Change release date to 2008, and version to 1.0 (since is GA). Signed-off-by: Jack Morgenstein --- drivers/net/mlx4/mlx4.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a4023c2..829c419 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -47,8 +47,8 @@ #define DRV_NAME "mlx4_core" #define PFX DRV_NAME ": " -#define DRV_VERSION "0.01" -#define DRV_RELDATE "May 1, 2007" +#define DRV_VERSION "1.0" +#define DRV_RELDATE "April 4, 2008" enum { MLX4_HCR_BASE = 0x80680, -- 1.5.1.6 From anuj01 at gmail.com Tue Jul 15 00:10:50 2008 From: anuj01 at gmail.com (=?UTF-8?B?4KSF4KSo4KWB4KSc?=) Date: Tue, 15 Jul 2008 12:40:50 +0530 Subject: [ofa-general] Hi - Uverbs flow path Message-ID: On Tue, Jul 15, 2008 at 12:29 PM, Roland Dreier wrote: > > I hav already downloaded the .tar file of libmthca and made changes to mthca.c. > > > > How can i rebuild this library and install it? > > ./configure && make install But it's not working. The output for ./configure is : ./configure: line 90: conf9108.sh: No such file or directory ./configure: line 91: conf9108.sh: No such file or directory chmod: cannot access `conf9108.sh': No such file or directory ./configure: line 203: conf9108.file: No such file or directory ./configure: line 1173: config.log: No such file or directory And every time I run ./configure the number 9108 in the above ouput increases to any unknown no. Why it's happening? n Which files r these? From Robert at saq.co.uk Tue Jul 15 00:24:39 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Tue, 15 Jul 2008 08:24:39 +0100 Subject: [ofa-general] Infinband and DRBD References: <2f3bf9a60807122324j60e48a9gddf3b01612c20d0b@mail.gmail.com> Message-ID: Hi, I think it answers my question. I'm curious to know IPOIB-CM would offer better performance but can't find much information on the difference between IPOIB and IPOIB-CM on Google. Does anyone have a link or a little bit of info on the differences? >From the little bit of info I can find it seems Centos 5.2 uses some sort of OFED 1.3 but I'm not entirely sure. Rob -----Original Message----- From: Dotan Barak [mailto:dotanba at gmail.com] Sent: 13 July 2008 07:25 To: Robert Dunkley Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] Infinband and DRBD I don't know which OFED version is installed in the Distribution version that you are using, but i think (as a general rule, at least at this stage): users should install the latest OFED release if this version number is higher than the one that originally being installed with your distribution. I hope that i answered your answer ... Dotan On Wed, Jul 9, 2008 at 5:12 PM, Robert Dunkley wrote: > Hi Everyone, > > > > Apologies if this question is a bit simple but if anyone has any thoughts > I'd appreciated them. > > > > This is my first Infiniband experiment. I need to clone disk partitions > between Linux boxes, theoretically 400-500Mbytes/second writes could be seen > at peak (Total for all partitions) but loading will mostly be smallish > random writes. I was thinking of just using Centos with the built-in IPOIB > support and DRBD but does anyone have any better suggestions? > > > > Thanks, > > > > Rob > > > > The SAQ Group > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > SEMTEC Limited Trading as SAQ is Registered in England & Wales > Company Number: 06481952 > > > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication and I.T. > services to UK Business. > > DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups > : Managed Networks : Remote Support. > > > > Find us in http://www.thebestof.co.uk/petersfield > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield From olaf.kirch at oracle.com Tue Jul 15 00:27:42 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Tue, 15 Jul 2008 09:27:42 +0200 Subject: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support In-Reply-To: <005b01c8e63c$4492d980$3edc10ac@YOURA06808D9DE> References: <20080703213412.GB32285@opengridcomputing.com> <200807081400.17025.olaf.kirch@oracle.com> <005b01c8e63c$4492d980$3edc10ac@YOURA06808D9DE> Message-ID: <200807150927.44040.olaf.kirch@oracle.com> On Tuesday 15 July 2008 07:33:05 Richard Frank wrote: > This all seems to be a bit restrictive - which will require some changes in > how Oracle uses the IPC for rdma operations - but we need to start somewhere > so this seems reasonable. The best we can do right now for iWARP is to require the app layer to use at least part of the cmsg interface. You can continue to use GET_MR and FREE_MR, but you have to tell the kernel about the r_key it's sending in a message by using CMSG_DEST. > If we can get bcopy running with Oracle - that'd be a great goal for 1.4. If I have the patches queued up, and will submit them to Vlad today. > it's going to require changes in Oracle to use IWARP RDMA - then we're > talking about a later Orace release to use it... > > I'm out on vacation will be back in two weeks.. Enjoy! Olaf > > Rick > > ----- Original Message ----- > From: "Olaf Kirch" > To: "Or Gerlitz" > Cc: ; > Sent: Tuesday, July 08, 2008 8:00 AM > Subject: Re: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support > > > > On Tuesday 08 July 2008 12:11:40 Or Gerlitz wrote: > >> Olaf Kirch wrote: > >> > Okay, there's a third way to fix this, which may even work with today's > >> > Oracle IPC code. The hack is to not submit the fastreg WR until we post > >> > a SEND that transmits the rkey to a peer. Then > >> > > >> > - we have a QP to post the WRs to > >> > - we post the fastreg and the SEND to the same QP, > >> > ensuring that the peer doesn't initiate the RDMA > >> > before we've set up the memory registration. > >> Yes, as Steve noted, using the same QP for the fastreg and send that > >> carries the rkey/stag is a must since the send should be fenced on the > >> fastreg completion to make sure the remote side uses a valid key. > > > > One theoretical problem is what happens if the application wants to > > register a memory range and send the rkey to two peers at the same time? > > We would have to stall the second send explicitly as long as we > > haven't seen the completion for the first send. > > > > Olaf > > -- > > Olaf Kirch | --- o --- Nous sommes du soleil we love when we play > > okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From vlad at lists.openfabrics.org Tue Jul 15 02:45:11 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 15 Jul 2008 02:45:11 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080715-0200 daily build status Message-ID: <20080715094511.32906E609E0@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26-rc6 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ia64_check/kernel_addons/backport/2.6.24/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:58: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.c:32: include/linux/slab.h:270: error: conflicting types for 'kzalloc' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ppc64_check/kernel_addons/backport/2.6.24/include/linux/kobject.h:74: error: previous implicit declaration of 'kzalloc' was here make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ppc64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080715-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From tziporet at dev.mellanox.co.il Tue Jul 15 03:52:28 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 15 Jul 2008 13:52:28 +0300 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> <487B8727.20003@opengridcomputing.com> <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> Message-ID: <487C816C.7040301@mellanox.co.il> Jeff Squyres wrote: > Ditto -- I got pulled in a few different directions and I couldn't > make today's call; sorry. > > - OMPI v1.3 > - What is the proposed release schedule for OFED 1.4? Alpha - next week and Beta by end of July. > - We do not yet have beta tarballs for for OMPI v1.3, but we might > be able to get some in time for the OFED v1.4 schedule If you can have something in the coming two weeks it will be good. > > - Question about IBCM for OFED v1.4: > - Any estimate on the following ticket: > https://bugs.openfabrics.org/show_bug.cgi?id=963 > - Any estimate on the "ibcm_listen() sometimes failing" issue? (I > don't think there's a ticket filed against this yet) > > I don't think anything was done here - Sean? Tziporet From tziporet at mellanox.co.il Tue Jul 15 04:57:59 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Tue, 15 Jul 2008 14:57:59 +0300 Subject: [ofa-general] July 14 OFED meeting summary on OFED 1.4 features completeness status Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD1969DD@mtlexch01.mtl.com> OFED meeting summary for July 14, 2008 on 1.4 features completeness status ======================================================================== == Summary: ======== - Most of components are ready for the feature freeze of OFED 1.4 - We should have alpha release next week. Details: ======== OFED 1.4 feature readiness for the freeze: - Kernel base 2.6.27 (start with 2.6.26) - 2.6.26 done, will move to 2.6.27 when rc1 available - Multi-Protocol support, Eth and IB for ConnectX - all patches submitted, to be merged this week - NFS-RDMA - kernel 2.6.26 done. Distros backports not ready yet. Will start with SLES10 and then RHEL5.1 - iSer Target - on work; merge should start this week - New iWARP verbs (fast memory thru send queue, local invalidate send work requests, Read with invalidate) - done in 2.6.27, will need backports when we move - IPoIB (LRO and RSS) - LRO - in 2.6.27, RSS - patches to be sent this week - RDS iWARP support - In progress. Still need to debug fastreg usage, but first merge has been done into Olaf's tree. - SDP (GA) - patches were sent, merged to OFED tree - Raw QP support with ConnectX - under testing - uDAPL - socket cm for scalability - done. UD extensions will be done end of week. - OpenSM - code freeze of openSM can be later since it's a closed user level apps. - ibnetdiscover library - accelerate other tools. - Congestion Control - code in ibutils done; under testing - MVAPICH 1.1 - Should have tarball by end of the month - MVAPICH2 1.2 - Should have tarball by end of the month - OpenMPI 1.3 - do not yet have beta tarballs for OMPI v1.3. Might be able to get some for the OFED v1.4 - Backports status: - ipath driver - RHEL5, SLES10 and all kernels below 2.6.18 - Qlogic - to be completed by end of the week - NFSoRDMA - we need at least RHEL5 and SLES 10 - Jeff Backer - on work - EHCA: SLES10/SP1/SP2 - IBM - done Other topics: * Woody raised the question of Dev-Con at SC08. Conclusion was that we do not see a need for a special event, but a meeting for one or two hours of all developers that will come to SC08 is a good idea. * New server setup: Jeff updated us that the new server arrived. The request is to move to the new server in 1-2 weeks. Meetings minutes on the web: http://www.openfabrics.org/txt/documentation/linux/EWG_meeting_minutes/ Tziporet From 5huuhilo at research.telcordia.com Tue Jul 15 03:23:59 2008 From: 5huuhilo at research.telcordia.com (emerson kam) Date: Tue, 15 Jul 2008 10:23:59 +0000 Subject: [ofa-general] best performance Message-ID: <000701c8e673$075b421b$a6a25d95@gqanuxdx> " My order arrived yesterday via registered mail in good order THE WATCH IS BEAUTIFUL AND EVEN BETTER THAN I EXPECTED." Try it for yourself - u will be amazed!! - The worlds largest online retailer of luxury products, including: Rolex Sports Models Rolex Datejusts Breitling Cartier Porsche Design Dolce & Gabbana Dior Gucci Hermes Watches Patek Philippe Visit - www.addsok.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From linyao at ict.ac.cn Tue Jul 15 05:54:44 2008 From: linyao at ict.ac.cn (linyao) Date: Tue, 15 Jul 2008 20:54:44 +0800 Subject: [ofa-general] [SDP]How does buffer size affect SDP bandwidth performance? Message-ID: <200807152054442501828@ict.ac.cn> Hi, all When I use netperf to test SDP bandwidth performance, I was confused with a problem. My test results show that the bandwidth performance is better while the send and receive buffer size is smaller. How does it explain? How dose buffer size affect SDP bandwidth performance? The table below is the result of my test. The second column is the result with 16M send buffer and 8M receive buffer. The third column is the result with 256K send buffer and 256K receive buffer. The last column is the result with 16K send buffer and 87380B receive buffer. Package Size(Byte) Bandwidth(Mbps) 128 684.51 682.55 725.43 256 1095.85 979.69 945.82 512 1498.62 1527.56 2474.6 1024 3191.12 4210.58 5147.91 2048 3994.1 5356.29 7212.09 4096 4546.15 6208.94 8127.58 8192 5962.7 7260.09 8621.43 16384 5100.91 6364.46 6575.32 32768 7446.83 6870.21 8889.46 65536 6617.12 7044.84 8992.61 131072 6587.27 6909.3 8867.24 My test is on two nodes. Each node is a DELL SC430 PC server, which has one 2.8GHz Intel Pentium-4 processor and 1GB DDR II-400 main Memory. These two nodes are interconnected with both Gigabit Ethernet and Mellanox In?niBand DDR HCAs. Each node runs RedHat AS 4.0 with Linux kernel 2.6.20. I use OpenFabrics OFED-1.2 and Firmware 1.2.0. Netperf has two options "-s" and "-S" to set the size of socket send buffer and socket receive buffer. The former is for local and the latter is for remote. I used these two options to set different size values for socket send and receive buffer in the local and remote. I think they represent the SDP send and receive buffer size, right? Any reply would be appreciated! Thank you! 2008-07-15 linyao -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmorreale at novell.com Tue Jul 15 06:07:45 2008 From: pmorreale at novell.com (Peter W. Morreale) Date: Tue, 15 Jul 2008 07:07:45 -0600 Subject: [ofa-general] IB verbs client/server programming... In-Reply-To: <2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com> References: <1216054280.15841.28.camel@hermosa.morrealenet> <2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com> Message-ID: <1216127265.15841.67.camel@hermosa.morrealenet> On Tue, 2008-07-15 at 09:39 +0300, Dotan Barak wrote: > > In other words, what interfaces do I have to look at to implement the > > "blocking" portion equivalent of a recv(2)? > Do you want to block until the data will be received? Yes. Right now, all the examples I see involve a tight polling loop waiting for, and acknowledging the events. > (if so: what is the meaning of this when the sender uses RDMA?) For my purposes, RDMA is merely another transport. > > What exactly to you want to do? > > > I'm creating a distributed memory system that will maintain data consistent chunks of memory on connected nodes. The system has control messages as well as the memory hunks to transport across the wire. What I want to wind up with is a one-to-one correspondence between memory hunks (that are variably sized) and "transport handles". Along with another handle for control msgs. At least, I think I want that. :-) Right now I'm trying to shoe-horn in both control msgs and data "msgs" into the same transport abstraction. This is proving difficult with the dearth of documentation for RDMA. I'm using RDMA-CM as well as (obviously) the IB verbs. I suppose I could move back to a socket interface for the control messages and reserve RDMA for the big data movement.... On a related note... Is it possible to reuse a cma_id for different memory regions by appropriately calling ibv_reg_mr() and ibv_dereg_mr()? Thanks so much. -PWM From linyao at ict.ac.cn Tue Jul 15 06:11:59 2008 From: linyao at ict.ac.cn (linyao) Date: Tue, 15 Jul 2008 21:11:59 +0800 Subject: [ofa-general] [SDP]How does buffer size affect SDP bandwidth performance? Message-ID: <200807152111593122182@ict.ac.cn> Hi, all When I use netperf to test SDP bandwidth performance, I was confused with a problem. My test results show that the bandwidth performance is better while the send and receive buffer size is smaller. How does it explain? How dose buffer size affect SDP bandwidth performance? The table below is the result of my test. The second column is the result with 16M send buffer and 8M receive buffer. The third column is the result with 256K send buffer and 256K receive buffer. The last column is the result with 16K send buffer and 87380B receive buffer. Package Size(Byte) Bandwidth(Mbps) 128 684.51 682.55 725.43 256 1095.85 979.69 945.82 512 1498.62 1527.56 2474.6 1024 3191.12 4210.58 5147.91 2048 3994.1 5356.29 7212.09 4096 4546.15 6208.94 8127.58 8192 5962.7 7260.09 8621.43 16384 5100.91 6364.46 6575.32 32768 7446.83 6870.21 8889.46 65536 6617.12 7044.84 8992.61 131072 6587.27 6909.3 8867.24 My test is on two nodes. Each node is a DELL SC430 PC server, which has one 2.8GHz Intel Pentium-4 processor and 1GB DDR II-400 main Memory. These two nodes are interconnected with both Gigabit Ethernet and Mellanox In?niBand DDR HCAs. Each node runs RedHat AS 4.0 with Linux kernel 2.6.20. I use OpenFabrics OFED-1.2 and Firmware 1.2.0. Netperf has two options "-s" and "-S" to set the size of socket send buffer and socket receive buffer. The former is for local and the latter is for remote. I used these two options to set different size values for socket send and receive buffer in the local and remote. I think they represent the SDP send and receive buffer size, right? Any reply would be appreciated! Thank you! 2008-07-15 linyao From dotanba at gmail.com Tue Jul 15 07:07:21 2008 From: dotanba at gmail.com (Dotan Barak) Date: Tue, 15 Jul 2008 17:07:21 +0300 Subject: [ofa-general] IB verbs client/server programming... In-Reply-To: <1216127265.15841.67.camel@hermosa.morrealenet> References: <1216054280.15841.28.camel@hermosa.morrealenet> <2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com> <1216127265.15841.67.camel@hermosa.morrealenet> Message-ID: <2f3bf9a60807150707g4323b997t9914bec19613b5ab@mail.gmail.com> On Tue, Jul 15, 2008 at 4:07 PM, Peter W. Morreale wrote: > On Tue, 2008-07-15 at 09:39 +0300, Dotan Barak wrote: > >> > In other words, what interfaces do I have to look at to implement the >> > "blocking" portion equivalent of a recv(2)? >> Do you want to block until the data will be received? > > Yes. Right now, all the examples I see involve a tight polling loop > waiting for, and acknowledging the events. > You need to poll for completion on the receive queue (IF you use opcode that will consume WR in the recieve queue at the receiver) > >> (if so: what is the meaning of this when the sender uses RDMA?) > > For my purposes, RDMA is merely another transport. RC/UC/UD are the transports, the operations (opcode) that you use define the behaviour (if the receiver is aware for incoming data or not). > > >> >> What exactly to you want to do? >> >> > > > I'm creating a distributed memory system that will maintain data > consistent chunks of memory on connected nodes. The system has control > messages as well as the memory hunks to transport across the wire. > > What I want to wind up with is a one-to-one correspondence between > memory hunks (that are variably sized) and "transport handles". Along > with another handle for control msgs. At least, I think I want > that. :-) > > Right now I'm trying to shoe-horn in both control msgs and data "msgs" > into the same transport abstraction. This is proving difficult with the > dearth of documentation for RDMA. > > I'm using RDMA-CM as well as (obviously) the IB verbs. > > I suppose I could move back to a socket interface for the control > messages and reserve RDMA for the big data movement.... You can do it over the verbs, you don't really need to use socket for this. > > > On a related note... Is it possible to reuse a cma_id for different > memory regions by appropriately calling ibv_reg_mr() and > ibv_dereg_mr()? Sorry, but i don't really know the answer for this. You should contact Sean Hefty for questions about the cma ... > > Thanks so much. > > -PWM Dotan From dotanba at gmail.com Tue Jul 15 00:35:54 2008 From: dotanba at gmail.com (Dotan Barak) Date: Tue, 15 Jul 2008 10:35:54 +0300 Subject: ***SPAM*** Re: [ofa-general] Infinband and DRBD In-Reply-To: References: <2f3bf9a60807122324j60e48a9gddf3b01612c20d0b@mail.gmail.com> Message-ID: <2f3bf9a60807150035u1ed2c1adl253b91795d746a8c@mail.gmail.com> On Tue, Jul 15, 2008 at 10:24 AM, Robert Dunkley wrote: > Hi, > > I think it answers my question. I'm curious to know IPOIB-CM would offer > better performance but can't find much information on the difference > between IPOIB and IPOIB-CM on Google. Does anyone have a link or a > little bit of info on the differences? (sorry but i don't have the numbers for you) > > From the little bit of info I can find it seems Centos 5.2 uses some > sort of OFED 1.3 but I'm not entirely sure. I think that CentOS 5.2 contains OFED 1.3. In the OpenFabrics web site you can download OFED 1.3.1 (which is a little bit better than 1.3). BUT I think that OFED 1.3 is good enough for your needs ... Dotan > > Rob > > -----Original Message----- > From: Dotan Barak [mailto:dotanba at gmail.com] > Sent: 13 July 2008 07:25 > To: Robert Dunkley > Cc: general at lists.openfabrics.org > Subject: Re: [ofa-general] Infinband and DRBD > > I don't know which OFED version is installed in the Distribution > version that you are using, > but i think (as a general rule, at least at this stage): users should > install the latest OFED > release if this version number is higher than the one that originally > being installed with > your distribution. > > I hope that i answered your answer ... > Dotan > > On Wed, Jul 9, 2008 at 5:12 PM, Robert Dunkley wrote: >> Hi Everyone, >> >> >> >> Apologies if this question is a bit simple but if anyone has any > thoughts >> I'd appreciated them. >> >> >> >> This is my first Infiniband experiment. I need to clone disk > partitions >> between Linux boxes, theoretically 400-500Mbytes/second writes could > be seen >> at peak (Total for all partitions) but loading will mostly be smallish >> random writes. I was thinking of just using Centos with the built-in > IPOIB >> support and DRBD but does anyone have any better suggestions? >> >> >> >> Thanks, >> >> >> >> Rob >> >> >> >> The SAQ Group >> >> Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ >> SEMTEC Limited Trading as SAQ is Registered in England & Wales >> Company Number: 06481952 >> >> >> >> http://www.saqnet.co.uk AS29219 >> >> SAQ Group Delivers high quality, honestly priced communication and > I.T. >> services to UK Business. >> >> DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : > Backups >> : Managed Networks : Remote Support. >> >> >> >> Find us in http://www.thebestof.co.uk/petersfield >> >> >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > > The SAQ Group > Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ > SEMTEC Limited trading as SAQ is Registered in England & Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. > DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. > > Find us in http://www.thebestof.co.uk/petersfield > > From amirv at mellanox.co.il Tue Jul 15 07:15:34 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Tue, 15 Jul 2008 17:15:34 +0300 Subject: [ofa-general] [PATCH] CMA: Notify ULP when CM exits the TIMEWAIT state. Message-ID: <12161313341500-git-send-email-amirv@mellanox.co.il> Signed-off-by: Amir Vadai --- drivers/infiniband/core/cma.c | 7 ++++++- include/rdma/rdma_cm.h | 3 ++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index ae11d5c..9fa1371 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -914,7 +914,10 @@ static int cma_ib_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) struct rdma_cm_event event; int ret = 0; - if (cma_disable_callback(id_priv, CMA_CONNECT)) + if ((ib_event->event != IB_CM_TIMEWAIT_EXIT && + cma_disable_callback(id_priv, CMA_CONNECT)) || + (ib_event->event == IB_CM_TIMEWAIT_EXIT && + cma_disable_callback(id_priv, CMA_DISCONNECT))) return 0; memset(&event, 0, sizeof event); @@ -950,6 +953,8 @@ static int cma_ib_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event) event.event = RDMA_CM_EVENT_DISCONNECTED; break; case IB_CM_TIMEWAIT_EXIT: + event.event = RDMA_CM_EVENT_TIMWAIT_EXIT; + break; case IB_CM_MRA_RECEIVED: /* ignore event */ goto out; diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h index 22bb2e7..80170bd 100644 --- a/include/rdma/rdma_cm.h +++ b/include/rdma/rdma_cm.h @@ -57,7 +57,8 @@ enum rdma_cm_event_type { RDMA_CM_EVENT_DISCONNECTED, RDMA_CM_EVENT_DEVICE_REMOVAL, RDMA_CM_EVENT_MULTICAST_JOIN, - RDMA_CM_EVENT_MULTICAST_ERROR + RDMA_CM_EVENT_MULTICAST_ERROR, + RDMA_CM_EVENT_TIMWAIT_EXIT }; enum rdma_port_space { -- 1.5.3 From john.russo at qlogic.com Tue Jul 15 07:20:28 2008 From: john.russo at qlogic.com (John Russo) Date: Tue, 15 Jul 2008 09:20:28 -0500 Subject: [ofa-general] Question Message-ID: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org> I apologize if I missed this when it occurred... Between 1.3 and 1.3.1 why were the chkconfig values (for RedHat) changed in the spec file from 15 85 to 05 95 and (for SLES) the requirement for network to be a required-start removed? __________________________ John F. Russo Manager, Engineering QLogic Corporation 780 Fifth Avenue, Suite 140 King of Prussia, PA 19406 Direct: 610-233-4866 Main: 610-233-4800 Fax: 610-233-4777 Cell: 610-246-9903 Email: John.Russo at qlogic.com www.qlogic.com True success is the undeniable truth that we have proved ourselves. -Joe Luppino-Esposito -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 3677 bytes Desc: image001.jpg URL: From amirv at mellanox.co.il Tue Jul 15 07:20:51 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Tue, 15 Jul 2008 17:20:51 +0300 Subject: [ofa-general] cma: add support for TIMEWAIT_EXIT event (second try) In-Reply-To: <000301c8e5d6$f7e82f50$9d37170a@amr.corp.intel.com> References: <48590FF2.8090507@mellanox.co.il> <001201c8dc74$2b8c6f40$b2fc070a@amr.corp.intel.com> <5D49E7A8952DC44FB38C38FA0D758EAD14E87F@mtlexch01.mtl.com> <000301c8e5d6$f7e82f50$9d37170a@amr.corp.intel.com> Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD196B1F@mtlexch01.mtl.com> I sent the patch to the list - and also attached the patch mail here. Thanks, - Amir > -----Original Message----- > From: Sean Hefty [mailto:sean.hefty at intel.com] > Sent: Monday, July 14, 2008 8:28 PM > To: Amir Vadai > Cc: general at lists.openfabrics.org; Oren Duer; Tziporet Koren > Subject: RE: [ofa-general] cma: add support for TIMEWAIT_EXIT > event (second try) > > >Acquiring the handler_mutex explicitly sounds like a good idea. > > Based on Roland's feedback to a patch from Or, keep your > original code with > cma_disable_remove() in the if statement. > > >When will it be commited? Will we have it in 2.6.27? > > That's up to Roland, but the changes look fine to me. I've > already modified the > librdmacm to account for this event, so getting it in 2.6.27 > would be my > preference. > > Can you re-send your original patch against Roland's > for-2.6.27 branch? > > - Sean > > -------------- next part -------------- An embedded message was scrubbed... From: "Amir Vadai" Subject: [PATCH] CMA: Notify ULP when CM exits the TIMEWAIT state. Date: Tue, 15 Jul 2008 17:15:34 +0300 Size: 2288 URL: From fenkes at de.ibm.com Tue Jul 15 07:27:44 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Tue, 15 Jul 2008 16:27:44 +0200 Subject: [ofa-general] [PATCH] IB/ehca: Backport for RHEL 4.6 Message-ID: <200807151627.44553.fenkes@de.ibm.com> Also, fix atomic64_t backport on powerpc64. Signed-off-by: Joachim Fenkes --- .../2.6.9_U6/include/asm-powerpc/atomic64.h | 61 +++++++ .../backport/2.6.9_U6/include/asm/atomic.h | 4 + .../2.6.9_U6/ehca_01_ibmebus_loc_code.patch | 186 ++++++++++++++++++++ 3 files changed, 251 insertions(+), 0 deletions(-) create mode 100644 kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h create mode 100644 kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch diff --git a/kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h b/kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h new file mode 100644 index 0000000..8036dbe --- /dev/null +++ b/kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h @@ -0,0 +1,61 @@ +#ifndef __ATOMIC64_INC_BACKPORT_2_6_9__ +#define __ATOMIC64_INC_BACKPORT_2_6_9__ + +typedef struct { volatile long counter; } atomic64_t; + +#define ATOMIC64_INIT(i) { (i) } + +#define atomic64_read(v) ((v)->counter) +#define atomic64_set(v,i) (((v)->counter) = (i)) + +static __inline__ void atomic64_add(long a, atomic64_t *v) +{ + long t; + + __asm__ __volatile__( +"1: ldarx %0,0,%3 # atomic64_add\n\ + add %0,%2,%0\n\ + stdcx. %0,0,%3 \n\ + bne- 1b" +: "=&r" (t), "=m" (v->counter) +: "r" (a), "r" (&v->counter), "m" (v->counter) +: "cc"); +} + +static __inline__ long atomic64_add_return(long a, atomic64_t *v) +{ + long t; + + __asm__ __volatile__( +#ifdef CONFIG_SMP +" lwsync\n" +#endif +"1: ldarx %0,0,%2 # atomic64_add_return\n\ + add %0,%1,%0\n\ + stdcx. %0,0,%2 \n\ + bne- 1b" +#ifdef CONFIG_SMP +" isync\n" +#endif + : "=&r" (t) + : "r" (a), "r" (&v->counter) + : "cc", "memory"); + + return t; +} + +static __inline__ void atomic64_inc(atomic64_t *v) +{ + long t; + + __asm__ __volatile__( +"1: ldarx %0,0,%2 # atomic64_inc\n\ + addic %0,%0,1\n\ + stdcx. %0,0,%2 \n\ + bne- 1b" +: "=&r" (t), "=m" (v->counter) +: "r" (&v->counter), "m" (v->counter) +: "cc"); +} + +#endif diff --git a/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h b/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h index 11de648..2823278 100644 --- a/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h +++ b/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h @@ -3,6 +3,10 @@ #include_next +#ifdef __powerpc64__ +#include +#endif + # if defined(__x86_64__) /** diff --git a/kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch b/kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch new file mode 100644 index 0000000..8737b79 --- /dev/null +++ b/kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch @@ -0,0 +1,186 @@ +diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h +index 1e9e99a..fbf6e58 100644 +--- a/drivers/infiniband/hw/ehca/ehca_classes.h ++++ b/drivers/infiniband/hw/ehca/ehca_classes.h +@@ -112,7 +112,7 @@ struct ehca_sport { + + struct ehca_shca { + struct ib_device ib_device; +- struct of_device *ofdev; ++ struct ibmebus_dev *ibmebus_dev; + u8 num_ports; + int hw_level; + struct list_head shca_list; +diff --git a/drivers/infiniband/hw/ehca/ehca_eq.c b/drivers/infiniband/hw/ehca/ehca_eq.c +index 49660df..aab80e9 100644 +--- a/drivers/infiniband/hw/ehca/ehca_eq.c ++++ b/drivers/infiniband/hw/ehca/ehca_eq.c +@@ -122,7 +122,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + /* register interrupt handlers and initialize work queues */ + if (type == EHCA_EQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_eq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_eq, + IRQF_DISABLED, "ehca_eq", + (void *)shca); + if (ret < 0) +@@ -130,7 +130,7 @@ int ehca_create_eq(struct ehca_shca *shca, + + tasklet_init(&eq->interrupt_task, ehca_tasklet_eq, (long)shca); + } else if (type == EHCA_NEQ) { +- ret = ibmebus_request_irq(eq->ist, ehca_interrupt_neq, ++ ret = ibmebus_request_irq(NULL, eq->ist, ehca_interrupt_neq, + IRQF_DISABLED, "ehca_neq", + (void *)shca); + if (ret < 0) +@@ -170,7 +170,7 @@ int ehca_destroy_eq(struct ehca_shca *shca, struct ehca_eq *eq) + u64 h_ret; + + spin_lock_irqsave(&eq->spinlock, flags); +- ibmebus_free_irq(eq->ist, (void *)shca); ++ ibmebus_free_irq(NULL, eq->ist, (void *)shca); + + h_ret = hipz_h_destroy_eq(shca->ipz_hca_handle, eq); + +diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c +index 482103e..666aafe 100644 +--- a/drivers/infiniband/hw/ehca/ehca_main.c ++++ b/drivers/infiniband/hw/ehca/ehca_main.c +@@ -289,8 +289,8 @@ static int ehca_sense_attributes(struct + }; + + ehca_gen_dbg("Probing adapter %s...", +- shca->ofdev->node->full_name); +- loc_code = of_get_property(shca->ofdev->node, "ibm,loc-code", NULL); ++ shca->ibmebus_dev->ofdev.node->full_name); ++ loc_code = of_get_property(shca->ibmebus_dev->ofdev.node, "ibm,loc-code", NULL); + if (loc_code) + ehca_gen_dbg(" ... location lode=%s", loc_code); + +@@ -458,7 +458,7 @@ static int ehca_init_device(struct ehca_shca *shca) + shca->ib_device.node_type = RDMA_NODE_IB_CA; + shca->ib_device.phys_port_cnt = shca->num_ports; + shca->ib_device.num_comp_vectors = 1; +- shca->ib_device.dma_device = &shca->ofdev->dev; ++ shca->ib_device.dma_device = &shca->ibmebus_dev->ofdev.dev; + shca->ib_device.query_device = ehca_query_device; + shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_gid = ehca_query_gid; +@@ -619,11 +619,6 @@ static struct attribute_group ehca_drv_attr_grp = { + .attrs = ehca_drv_attrs + }; + +-static struct attribute_group *ehca_drv_attr_groups[] = { +- &ehca_drv_attr_grp, +- NULL, +-}; +- + #define EHCA_RESOURCE_ATTR(name) \ + static ssize_t ehca_show_##name(struct device *dev, \ + struct device_attribute *attr, \ +@@ -707,7 +702,7 @@ static struct attribute_group ehca_dev_attr_grp = { + .attrs = ehca_dev_attrs + }; + +-static int __devinit ehca_probe(struct of_device *dev, ++static int __devinit ehca_probe(struct ibmebus_dev *dev, + const struct of_device_id *id) + { + struct ehca_shca *shca; +@@ -715,16 +710,16 @@ static int __devinit ehca_probe(struct of_device *dev, + struct ib_pd *ibpd; + int ret, i, eq_size; + +- handle = of_get_property(dev->node, "ibm,hca-handle", NULL); ++ handle = of_get_property(dev->ofdev.node, "ibm,hca-handle", NULL); + if (!handle) { + ehca_gen_err("Cannot get eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + + if (!(*handle)) { + ehca_gen_err("Wrong eHCA handle for adapter: %s.", +- dev->node->full_name); ++ dev->ofdev.node->full_name); + return -ENODEV; + } + +@@ -739,9 +734,9 @@ static int __devinit ehca_probe(struct of_device *dev, + for (i = 0; i < ARRAY_SIZE(shca->sport); i++) + spin_lock_init(&shca->sport[i].mod_sqp_lock); + +- shca->ofdev = dev; ++ shca->ibmebus_dev = dev; + shca->ipz_hca_handle.handle = *handle; +- dev->dev.driver_data = shca; ++ dev->ofdev.dev.driver_data = shca; + + ret = ehca_sense_attributes(shca); + if (ret < 0) { +@@ -818,7 +813,7 @@ static int __devinit ehca_probe(struct of_device *dev, + } + } + +- ret = sysfs_create_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ ret = sysfs_create_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + if (ret) /* only complain; we can live without attributes */ + ehca_err(&shca->ib_device, + "Cannot create device attributes ret=%d", ret); +@@ -868,12 +863,12 @@ probe1: + return -EINVAL; + } + +-static int __devexit ehca_remove(struct of_device *dev) ++static int __devexit ehca_remove(struct ibmebus_dev *dev) + { +- struct ehca_shca *shca = dev->dev.driver_data; ++ struct ehca_shca *shca = dev->ofdev.dev.driver_data; + int ret; + +- sysfs_remove_group(&dev->dev.kobj, &ehca_dev_attr_grp); ++ sysfs_remove_group(&dev->ofdev.dev.kobj, &ehca_dev_attr_grp); + + if (ehca_open_aqp1 == 1) { + int i; +@@ -924,14 +919,11 @@ static struct of_device_id ehca_device_table[] = + {}, + }; + +-static struct of_platform_driver ehca_driver = { +- .name = "ehca", +- .match_table = ehca_device_table, +- .probe = ehca_probe, +- .remove = ehca_remove, +- .driver = { +- .groups = ehca_drv_attr_groups, +- }, ++static struct ibmebus_driver ehca_driver = { ++ .name = "ehca", ++ .id_table = ehca_device_table, ++ .probe = ehca_probe, ++ .remove = ehca_remove, + }; + + void ehca_poll_eqs(unsigned long data) +@@ -990,6 +982,10 @@ static int __init ehca_module_init(void) + goto module_init2; + } + ++ ret = sysfs_create_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); ++ if (ret) /* only complain; we can live without attributes */ ++ ehca_gen_err("Cannot create driver attributes ret=%d", ret); ++ + if (ehca_poll_all_eqs != 1) { + ehca_gen_err("WARNING!!!"); + ehca_gen_err("It is possible to lose interrupts."); +@@ -1015,6 +1011,7 @@ static void __exit ehca_module_exit(void) + if (ehca_poll_all_eqs == 1) + del_timer_sync(&poll_eqs_timer); + ++ sysfs_remove_group(&ehca_driver.driver.kobj, &ehca_drv_attr_grp); + ibmebus_unregister_driver(&ehca_driver); + + ehca_destroy_slab_caches(); + -- 1.5.5 From jackm at dev.mellanox.co.il Tue Jul 15 07:45:52 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 15 Jul 2008 17:45:52 +0300 Subject: [ofa-general] [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) Message-ID: <200807151745.53152.jackm@dev.mellanox.co.il> From 01d8c13b6dd314b239a5b0b76e57320aeaa5c338 Mon Sep 17 00:00:00 2001 From: Jack Morgenstein Date: Tue, 15 Jul 2008 17:19:41 +0300 Subject: [PATCH] mlx4_core: add module parameter to enable/disable QoS support. Add a module parameter "enable_qos" to mlx4_core. If this param is > 0, support for QoS is enabled in the HCA (INIT_HCA command). By default, the parameter is set to 0 (disabled). Signed-off-by: Jack Morgenstein --- drivers/net/mlx4/fw.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index 2b5006b..2b60525 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -46,6 +46,10 @@ enum { extern void __buggy_use_of_MLX4_GET(void); extern void __buggy_use_of_MLX4_PUT(void); +static int mlx4_core_enable_qos; +module_param_named(enable_qos, mlx4_core_enable_qos, int, 0444); +MODULE_PARM_DESC(enable_qos, "Enable Quality of Service support in the HCA if > 0, (default 0)"); + #define MLX4_GET(dest, source, offset) \ do { \ void *__p = (char *) (source) + (offset); \ @@ -737,6 +741,10 @@ int mlx4_INIT_HCA(struct mlx4_dev *dev, struct mlx4_init_hca_param *param) if (dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 3); + /* Enable QoS support if module parameter set */ + if (mlx4_core_enable_qos) + *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 2); + /* QPC/EEC/CQC/EQC/RDMARC attributes */ MLX4_PUT(inbox, param->qpc_base, INIT_HCA_QPC_BASE_OFFSET); -- 1.5.1.6 From okir at lst.de Tue Jul 15 08:14:30 2008 From: okir at lst.de (Olaf Kirch) Date: Tue, 15 Jul 2008 17:14:30 +0200 Subject: [ofa-general] Re: [rds-devel] [PATCH RFC] rds: add iwarp support In-Reply-To: <200807040009.47794.olaf.kirch@oracle.com> References: <20080703213412.GB32285@opengridcomputing.com> <200807040009.47794.olaf.kirch@oracle.com> Message-ID: <200807151714.31106.okir@lst.de> Hi, I've made a new code drop available on my git tree. It has 90% of the RDMA code as I would like to do it. There's one major missing FIXME - the allocation of the WR array for the inval requests - I didn't get around to that today, and now I have to run. In general, we have this setup - the mr_pool gets a new set of function pointers so we can do fmrs or fastregs flexibly - there's a dummy qp associated with each mr pool that is used for inval requests. - there's a new structure called rds_ib_mapping, which has all the relevant details to handle the mapping side of a mr - if you allocate a fastreg mr, we will do the dma mapping etc. The MAP WR will be submitted when you send a message with CMSG_RDMA_DEST referring to this specific MR. - FREE_MR will do the old-fashioned batched unmap for fmrs, but for fastregs we will try to create a clone rds_ib_mapping that contains the s/g list, and the r_key. establishing and invalidating a mapping is decoupled, and happens on separate QPs - I think this is okay as the local_inv request is really just the r_key that we want to invalidate, so we can reuse the struct ib_mr as many times as we want as long as we bump the r_key. Okay, this is very sketchy - please go and look at the code, which should tell you more. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From vlad at mellanox.co.il Tue Jul 15 08:25:23 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 15 Jul 2008 18:25:23 +0300 Subject: [ofa-general] Re: [ewg] [PATCH 0/5] eHCA backports for OFED 1.4, part 1 In-Reply-To: <200807142128.53998.fenkes@de.ibm.com> References: <200807142128.53998.fenkes@de.ibm.com> Message-ID: <487CC163.3090604@mellanox.co.il> Joachim Fenkes wrote: > Hi, > > this patchset contains eHCA backports for several kernels as well as some > fixes to other backports that broke on powerpc. > > [1/5] fixes core for 2.6.24 > [2/5] fixes core for RHEL5.1 and RHEL 5.2 > [3/5] adds ehca backports for 2.6.2[345] > [4/5] adds ehca backports for RHEL5.1 and RHEL5.2 > [5/5] adds ehca backports for SLES10 SP1 and SP2 > > Please review the changes, especially the core changes, and apply if you > think they're okay. > > Thanks and regards! > Joachim and Alex > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg Applied 1-5, [2/5] added for RHEL5.0 too. Regards, Vladimir From vlad at mellanox.co.il Tue Jul 15 08:39:36 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 15 Jul 2008 18:39:36 +0300 Subject: [ofa-general] Question In-Reply-To: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org> References: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org> Message-ID: <487CC4B8.5080506@mellanox.co.il> John Russo wrote: > I apologize if I missed this when it occurred… > > > > Between 1.3 and 1.3.1 why were the chkconfig values (for RedHat) changed > in the spec file from 15 85 to 05 95 and (for SLES) the requirement for > network to be a required-start removed? > > > > * > ***__________________________*** > **John F. Russo** > **Manager, Engineering** > **QLogic Corporation** > **780 Fifth Avenue, Suite 140** > **King of Prussia****, PA 19406** > **Direct: 610-233-4866** > **Main****: 610-233-4800** > **Fax: 610-233-4777** > **Cell: 610-246-9903** > **Email: *****John.Russo at qlogic.com** > **www.qlogic.com** > This changes the start order of /etc/init.d/network and /etc/init.d/openibd scripts. Now the openibd script start first. And this enables bonding module to be configured correctly (after ib_ipoib module being loaded). Regards, Vladimir From vlad at mellanox.co.il Tue Jul 15 08:54:19 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 15 Jul 2008 18:54:19 +0300 Subject: [ofa-general] Re: [ewg] [PATCH] IB/ehca: Backport for RHEL 4.6 In-Reply-To: <200807151627.44553.fenkes@de.ibm.com> References: <200807151627.44553.fenkes@de.ibm.com> Message-ID: <487CC82B.1040501@mellanox.co.il> Joachim Fenkes wrote: > Also, fix atomic64_t backport on powerpc64. > > Signed-off-by: Joachim Fenkes > --- > .../2.6.9_U6/include/asm-powerpc/atomic64.h | 61 +++++++ > .../backport/2.6.9_U6/include/asm/atomic.h | 4 + > .../2.6.9_U6/ehca_01_ibmebus_loc_code.patch | 186 ++++++++++++++++++++ > 3 files changed, 251 insertions(+), 0 deletions(-) > create mode 100644 kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h > create mode 100644 kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch Applied, Should the same be done for 2.6.9_U4 and 2.6.9_U5? Regards. Vladimir From pmorreale at novell.com Tue Jul 15 08:55:11 2008 From: pmorreale at novell.com (Peter W. Morreale) Date: Tue, 15 Jul 2008 09:55:11 -0600 Subject: [ofa-general] IB verbs client/server programming... In-Reply-To: <2f3bf9a60807150707g4323b997t9914bec19613b5ab@mail.gmail.com> References: <1216054280.15841.28.camel@hermosa.morrealenet> <2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com> <1216127265.15841.67.camel@hermosa.morrealenet> <2f3bf9a60807150707g4323b997t9914bec19613b5ab@mail.gmail.com> Message-ID: <1216137311.15841.124.camel@hermosa.morrealenet> On Tue, 2008-07-15 at 17:07 +0300, Dotan Barak wrote: > On Tue, Jul 15, 2008 at 4:07 PM, Peter W. Morreale wrote: > > On Tue, 2008-07-15 at 09:39 +0300, Dotan Barak wrote: > > > >> > In other words, what interfaces do I have to look at to implement the > >> > "blocking" portion equivalent of a recv(2)? > >> Do you want to block until the data will be received? > > > > Yes. Right now, all the examples I see involve a tight polling loop > > waiting for, and acknowledging the events. > > > You need to poll for completion on the receive queue > (IF you use opcode that will consume WR in the recieve queue at the receiver) > I'm sorry, I don't understand this. What do you mean by "If you use opcode that will consume the work request in the receive queue..." What opcodes are you referring to? Are you saying that the receiver does not have to acknowledge the receipt of the data transfer? (That would be ideal for the data transfers.) If so, how do I set that up? The examples I have seen have the receiver ack'ing events. Can these be deferred (or ignored even?) Thanks, -PWM > > > > > >> (if so: what is the meaning of this when the sender uses RDMA?) > > > > For my purposes, RDMA is merely another transport. > RC/UC/UD are the transports, the operations (opcode) that you use > define the behaviour (if the receiver is aware for incoming data or > not). > > > > > > >> > >> What exactly to you want to do? > >> > >> > > > > > I'm creating a distributed memory system that will maintain data > > consistent chunks of memory on connected nodes. The system has control > > messages as well as the memory hunks to transport across the wire. > > > > What I want to wind up with is a one-to-one correspondence between > > memory hunks (that are variably sized) and "transport handles". Along > > with another handle for control msgs. At least, I think I want > > that. :-) > > > > Right now I'm trying to shoe-horn in both control msgs and data "msgs" > > into the same transport abstraction. This is proving difficult with the > > dearth of documentation for RDMA. > > > > I'm using RDMA-CM as well as (obviously) the IB verbs. > > > > I suppose I could move back to a socket interface for the control > > messages and reserve RDMA for the big data movement.... > You can do it over the verbs, you don't really need to use socket for this. > > > > > > > On a related note... Is it possible to reuse a cma_id for different > > memory regions by appropriately calling ibv_reg_mr() and > > ibv_dereg_mr()? > Sorry, but i don't really know the answer for this. > You should contact Sean Hefty for questions about the cma ... > > > > > Thanks so much. > > > > -PWM > > Dotan From eli at dev.mellanox.co.il Tue Jul 15 09:12:51 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 15 Jul 2008 19:12:51 +0300 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support Message-ID: <20080715161251.GA16299@mtls03> Following this email is a list of 3 patches which implements receive core affinity for IPoIB. In tests I made on my machine I got the following results with netperf TCP, 4 streams. Without RCA: 730 MB/s With RCA: 970 MB/s The patches require the patches sent by Yevegeni P. to provide support for allocation of multiple queues. I have sent an RFC a few weeks ago but it did not receive too much attention, so I implemented on the original API I sent back then. I would like this to be reviewed by the community and good comments are welcome. From eli at mellanox.co.il Tue Jul 15 09:13:31 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 15 Jul 2008 19:13:31 +0300 Subject: [ofa-general] [PATCH] IB/core: Add support for Receive Core Affinity Message-ID: <20080715161331.GA8667@mtls03> Add the capability flag IB_DEVICE_IPOIB_RCA to denote devices which support distribution of received packects to multiple receive queues. This results in better utilization of the system CPU cores by distributing interrupt handling between the cores. The patch adds a new verb, ib_create_qp_range(), to create a list range of QPs with specific alignment requirements that should be used by a consumer to for the different receive queues. Signed-off-by: Eli Cohen --- drivers/infiniband/core/verbs.c | 39 ++++++++++++++++++++++++++++++++++++++- include/rdma/ib_verbs.h | 30 +++++++++++++++++++++++++++++- 2 files changed, 67 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index a7da9be..871fb1e 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -280,6 +280,39 @@ EXPORT_SYMBOL(ib_destroy_srq); /* Queue pairs */ +int ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr *qp_init_attr, + int nqps, int align, struct ib_qp *list[]) +{ + int ret; + int i; + + ret = pd->device->create_qp_range ? + pd->device->create_qp_range(pd, qp_init_attr, NULL, nqps, + align, list) : + -ENOSYS; + + if (!ret) { + for (i = 0; i < nqps; ++i) { + list[i]->device = pd->device; + list[i]->pd = pd; + list[i]->send_cq = qp_init_attr[i].send_cq; + list[i]->recv_cq = qp_init_attr[i].recv_cq; + list[i]->srq = qp_init_attr[i].srq; + list[i]->uobject = NULL; + list[i]->event_handler = qp_init_attr[i].event_handler; + list[i]->qp_context = qp_init_attr[i].qp_context; + list[i]->qp_type = qp_init_attr[i].qp_type; + atomic_inc(&pd->usecnt); + atomic_inc(&qp_init_attr[i].send_cq->usecnt); + atomic_inc(&qp_init_attr[i].recv_cq->usecnt); + if (qp_init_attr[i].srq) + atomic_inc(&qp_init_attr[i].srq->usecnt); + } + } + return ret; +} +EXPORT_SYMBOL(ib_create_qp_range); + struct ib_qp *ib_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *qp_init_attr) { @@ -331,6 +364,9 @@ static const struct { IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), + }, + .opt_param = { + [IB_QPT_UD] = IB_QP_RCA, } }, }, @@ -427,7 +463,8 @@ static const struct { .valid = 1, .opt_param = { [IB_QPT_UD] = (IB_QP_CUR_STATE | - IB_QP_QKEY), + IB_QP_QKEY | + IB_QP_RCA), [IB_QPT_UC] = (IB_QP_CUR_STATE | IB_QP_ACCESS_FLAGS | IB_QP_ALT_PATH | diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 58a0c3f..3bb391b 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -105,6 +105,7 @@ enum ib_device_cap_flags { IB_DEVICE_UD_TSO = (1<<19), IB_DEVICE_MEM_MGT_EXTENSIONS = (1<<21), IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22), + IB_DEVICE_IPOIB_RCA = (1<<23), }; enum ib_atomic_cap { @@ -558,6 +559,7 @@ enum ib_qp_type { enum ib_qp_create_flags { IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + IB_QP_CREATE_IPOIB_RCA = 1 << 2, }; struct ib_qp_init_attr { @@ -629,7 +631,8 @@ enum ib_qp_attr_mask { IB_QP_MAX_DEST_RD_ATOMIC = (1<<17), IB_QP_PATH_MIG_STATE = (1<<18), IB_QP_CAP = (1<<19), - IB_QP_DEST_QPN = (1<<20) + IB_QP_DEST_QPN = (1<<20), + IB_QP_RCA = (1<<21) }; enum ib_qp_state { @@ -648,6 +651,12 @@ enum ib_mig_state { IB_MIG_ARMED }; +struct rca_attr { + int base_qpn; + int num_qpn; + int default_qpn; +}; + struct ib_qp_attr { enum ib_qp_state qp_state; enum ib_qp_state cur_qp_state; @@ -674,6 +683,7 @@ struct ib_qp_attr { u8 rnr_retry; u8 alt_port_num; u8 alt_timeout; + struct rca_attr rca; }; enum ib_wr_opcode { @@ -1043,6 +1053,10 @@ struct ib_device { int (*post_srq_recv)(struct ib_srq *srq, struct ib_recv_wr *recv_wr, struct ib_recv_wr **bad_recv_wr); + int (*create_qp_range)(struct ib_pd *pd, + struct ib_qp_init_attr *qp_init_attr, + struct ib_udata *udata, int nqps, + int align, struct ib_qp *list[]); struct ib_qp * (*create_qp)(struct ib_pd *pd, struct ib_qp_init_attr *qp_init_attr, struct ib_udata *udata); @@ -1373,6 +1387,20 @@ static inline int ib_post_srq_recv(struct ib_srq *srq, } /** + * ib_create_qp_range - Creates a range of QPs associated with the + * specified protection domain. + * @pd: The protection domain associated with the QP. + * @qp_init_attr: A list of initial attributes required to create the + * QPs. If QP creation succeeds, then the attributes are updated to + * the actual capabilities of the created QP. + * @nqps: The number of required QPs + * @align: Required alignment for the allocated QP numbers. + * @list: pointer to an array to hold allocated QPs + */ +int ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr *qp_init_attr, + int nqps, int align, struct ib_qp *list[]); + +/** * ib_create_qp - Creates a QP associated with the specified protection * domain. * @pd: The protection domain associated with the QP. -- 1.5.6 From eli at mellanox.co.il Tue Jul 15 09:14:00 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 15 Jul 2008 19:14:00 +0300 Subject: [ofa-general] [PATCH] IB/mlx4: Add support for Receive Affinity Message-ID: <20080715161400.GA8693@mtls03> This patch handles the implementation at the mlx4 level, of the changes required to support RCA. It mainly handles the creation of a range of QPs and also handles the configuration of the special RCA QP and the required changes to the inbox parameters. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/main.c | 4 +- drivers/infiniband/hw/mlx4/mlx4_ib.h | 4 + drivers/infiniband/hw/mlx4/qp.c | 228 +++++++++++++++++++++------------- include/linux/mlx4/qp.h | 48 +++++++- 4 files changed, 193 insertions(+), 91 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 0c453d0..d3c8878 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -91,7 +91,8 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, IB_DEVICE_PORT_ACTIVE_EVENT | IB_DEVICE_SYS_IMAGE_GUID | IB_DEVICE_RC_RNR_NAK_GEN | - IB_DEVICE_BLOCK_MULTICAST_LOOPBACK; + IB_DEVICE_BLOCK_MULTICAST_LOOPBACK | + IB_DEVICE_IPOIB_RCA; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_BAD_PKEY_CNTR) props->device_cap_flags |= IB_DEVICE_BAD_PKEY_CNTR; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_BAD_QKEY_CNTR) @@ -618,6 +619,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.query_srq = mlx4_ib_query_srq; ibdev->ib_dev.destroy_srq = mlx4_ib_destroy_srq; ibdev->ib_dev.post_srq_recv = mlx4_ib_post_srq_recv; + ibdev->ib_dev.create_qp_range = mlx4_ib_create_qp_range; ibdev->ib_dev.create_qp = mlx4_ib_create_qp; ibdev->ib_dev.modify_qp = mlx4_ib_modify_qp; ibdev->ib_dev.query_qp = mlx4_ib_query_qp; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index ec9bf28..e26c3d6 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -103,6 +103,7 @@ struct mlx4_ib_wq { enum mlx4_ib_qp_flags { MLX4_IB_QP_LSO = 1 << 0, MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + MLX4_IB_QP_RCA = 1 << 2, }; struct mlx4_ib_qp { @@ -268,6 +269,9 @@ void mlx4_ib_free_srq_wqe(struct mlx4_ib_srq *srq, int wqe_index); int mlx4_ib_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr, struct ib_recv_wr **bad_wr); +int mlx4_ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr *qp_init_attr, + struct ib_udata *udata, int nqps, + int align, struct ib_qp *list[]); struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *init_attr, struct ib_udata *udata); diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index fc61556..72a2d5d 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -502,9 +502,10 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, if (init_attr->create_flags & IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; - if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) qp->flags |= MLX4_IB_QP_LSO; + if (init_attr->create_flags & IB_QP_CREATE_IPOIB_RCA) + qp->flags |= MLX4_IB_QP_RCA; err = set_kernel_sq_size(dev, &init_attr->cap, init_attr->qp_type, qp); if (err) @@ -541,11 +542,6 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } } - if (!sqpn) - err = mlx4_qp_reserve_range(dev->dev, 1, 1, &sqpn); - if (err) - goto err_wrid; - err = mlx4_qp_alloc(dev->dev, sqpn, &qp->mqp); if (err) { mlx4_qp_release_range(dev->dev, sqpn, 1); @@ -659,9 +655,6 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp, mlx4_qp_free(dev->dev, &qp->mqp); - if (!is_sqp(dev, qp)) - mlx4_qp_release_range(dev->dev, qp->mqp.qpn, 1); - mlx4_mtt_cleanup(dev->dev, &qp->mtt); if (is_user) { @@ -678,91 +671,138 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp, } } -struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, - struct ib_qp_init_attr *init_attr, - struct ib_udata *udata) +int mlx4_ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr *init_attr, + struct ib_udata *udata, int nqps, + int align, struct ib_qp *list[]) { struct mlx4_ib_dev *dev = to_mdev(pd->device); struct mlx4_ib_sqp *sqp; struct mlx4_ib_qp *qp; int err; + int base_qpn, qpn; + int i; - /* - * We only support LSO and multicast loopback blocking, and - * only for kernel UD QPs. - */ - if (init_attr->create_flags & ~(IB_QP_CREATE_IPOIB_UD_LSO | - IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)) - return ERR_PTR(-EINVAL); + for (i = 0; i < nqps; ++i) { + /* + * We only support LSO, multicast loopback blocking and RCA, and + * only for kernel UD QPs. + */ + if (init_attr[i].create_flags & ~(IB_QP_CREATE_IPOIB_UD_LSO | + IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK | + IB_QP_CREATE_IPOIB_RCA)) + return -EINVAL; + if (init_attr[i].create_flags & (IB_QP_CREATE_IPOIB_UD_LSO | + IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK | + IB_QP_CREATE_IPOIB_RCA) && + (pd->uobject || init_attr[i].qp_type != IB_QPT_UD)) + return -EINVAL; - if (init_attr->create_flags && - (pd->uobject || init_attr->qp_type != IB_QPT_UD)) - return ERR_PTR(-EINVAL); + /* Userspace is not allowed to create special QPs: */ + if (pd->uobject && (init_attr[i].qp_type == IB_QPT_SMI || + init_attr[i].qp_type == IB_QPT_GSI)) + return -EINVAL; - switch (init_attr->qp_type) { - case IB_QPT_RC: - case IB_QPT_UC: - case IB_QPT_UD: - { - qp = kzalloc(sizeof *qp, GFP_KERNEL); - if (!qp) - return ERR_PTR(-ENOMEM); - - err = create_qp_common(dev, pd, init_attr, udata, 0, qp); - if (err) { - kfree(qp); - return ERR_PTR(err); - } + if (nqps > 1 && (init_attr[i].qp_type == IB_QPT_SMI || + init_attr[i].qp_type == IB_QPT_GSI)) + return -EINVAL; + } - qp->ibqp.qp_num = qp->mqp.qpn; + err = mlx4_qp_reserve_range(dev->dev, nqps, align, &base_qpn); + if (err) + return err; - break; - } - case IB_QPT_SMI: - case IB_QPT_GSI: - { - /* Userspace is not allowed to create special QPs: */ - if (pd->uobject) - return ERR_PTR(-EINVAL); - - sqp = kzalloc(sizeof *sqp, GFP_KERNEL); - if (!sqp) - return ERR_PTR(-ENOMEM); - - qp = &sqp->qp; - - err = create_qp_common(dev, pd, init_attr, udata, - dev->dev->caps.sqp_start + - (init_attr->qp_type == IB_QPT_SMI ? 0 : 2) + - init_attr->port_num - 1, - qp); - if (err) { - kfree(sqp); - return ERR_PTR(err); + for (i = 0, qpn = base_qpn; i < nqps; ++i, ++qpn) { + switch (init_attr[i].qp_type) { + case IB_QPT_RC: + case IB_QPT_UC: + case IB_QPT_UD: + { + qp = kzalloc(sizeof *qp, GFP_KERNEL); + if (!qp) { + err = -ENOMEM; + goto exit_fail; + } + + err = create_qp_common(dev, pd, init_attr + i, udata, qpn, qp); + if (err) { + kfree(qp); + err = err; + goto exit_fail; + } + + qp->ibqp.qp_num = qp->mqp.qpn; + + break; } + case IB_QPT_SMI: + case IB_QPT_GSI: + { + sqp = kzalloc(sizeof *sqp, GFP_KERNEL); + if (!sqp) { + err = -ENOMEM; + goto exit_fail; + } - qp->port = init_attr->port_num; - qp->ibqp.qp_num = init_attr->qp_type == IB_QPT_SMI ? 0 : 1; + qp = &sqp->qp; - break; - } - default: - /* Don't support raw QPs */ - return ERR_PTR(-EINVAL); + err = create_qp_common(dev, pd, init_attr + i, udata, + dev->dev->caps.sqp_start + + (init_attr[i].qp_type == IB_QPT_SMI ? 0 : 2) + + init_attr[i].port_num - 1, + qp); + if (err) { + kfree(sqp); + goto exit_fail; + } + + qp->port = init_attr[i].port_num; + qp->ibqp.qp_num = init_attr[i].qp_type == IB_QPT_SMI ? 0 : 1; + + break; + } + default: + /* Don't support raw QPs */ + err = -EINVAL; + goto exit_fail; + } + list[i] = &qp->ibqp; } + return 0; + +exit_fail: + for (--i; i >= 0; --i) + destroy_qp_common(dev, to_mqp(list[i]), !!pd->uobject); - return &qp->ibqp; + mlx4_qp_release_range(dev->dev, base_qpn, nqps); + return err; +} + +struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, + struct ib_qp_init_attr *init_attr, + struct ib_udata *udata) +{ + struct ib_qp *qp; + int err; + + err = mlx4_ib_create_qp_range(pd, init_attr, udata, 1, 1, &qp); + if (err) + return ERR_PTR(err); + + return qp; } int mlx4_ib_destroy_qp(struct ib_qp *qp) { struct mlx4_ib_dev *dev = to_mdev(qp->device); struct mlx4_ib_qp *mqp = to_mqp(qp); + int qpn = qp->qp_num; if (is_qp0(dev, mqp)) mlx4_CLOSE_PORT(dev->dev, mqp->port); destroy_qp_common(dev, mqp, !!qp->pd->uobject); + if (qpn >= dev->dev->caps.sqp_start + 8) + mlx4_qp_release_range(dev->dev, qpn, 1); if (is_sqp(dev, mqp)) kfree(to_msqp(mqp)); @@ -884,6 +924,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->flags = cpu_to_be32((to_mlx4_state(new_state) << 28) | (to_mlx4_st(ibqp->qp_type) << 16)); context->flags |= cpu_to_be32(1 << 8); /* DE? */ + context->flags |= cpu_to_be32(qp->flags & MLX4_IB_QP_RCA ? 1 << 13 : 0); if (!(attr_mask & IB_QP_PATH_MIG_STATE)) context->flags |= cpu_to_be32(MLX4_QP_PM_MIGRATED << 11); @@ -942,18 +983,18 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, if (attr_mask & IB_QP_PORT) { if (cur_state == IB_QPS_SQD && new_state == IB_QPS_SQD && !(attr_mask & IB_QP_AV)) { - mlx4_set_sched(&context->pri_path, attr->port_num); + mlx4_set_sched(&context->path.pri_path, attr->port_num); optpar |= MLX4_QP_OPTPAR_SCHED_QUEUE; } } if (attr_mask & IB_QP_PKEY_INDEX) { - context->pri_path.pkey_index = attr->pkey_index; + context->path.pri_path.pkey_index = attr->pkey_index; optpar |= MLX4_QP_OPTPAR_PKEY_INDEX; } if (attr_mask & IB_QP_AV) { - if (mlx4_set_path(dev, &attr->ah_attr, &context->pri_path, + if (mlx4_set_path(dev, &attr->ah_attr, &context->path.pri_path, attr_mask & IB_QP_PORT ? attr->port_num : qp->port)) goto out; @@ -962,7 +1003,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, } if (attr_mask & IB_QP_TIMEOUT) { - context->pri_path.ackto = attr->timeout << 3; + context->path.pri_path.ackto = attr->timeout << 3; optpar |= MLX4_QP_OPTPAR_ACK_TIMEOUT; } @@ -975,12 +1016,12 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, dev->dev->caps.pkey_table_len[attr->alt_port_num]) goto out; - if (mlx4_set_path(dev, &attr->alt_ah_attr, &context->alt_path, + if (mlx4_set_path(dev, &attr->alt_ah_attr, &context->path.alt_path, attr->alt_port_num)) goto out; - context->alt_path.pkey_index = attr->alt_pkey_index; - context->alt_path.ackto = attr->alt_timeout << 3; + context->path.alt_path.pkey_index = attr->alt_pkey_index; + context->path.alt_path.ackto = attr->alt_timeout << 3; optpar |= MLX4_QP_OPTPAR_ALT_ADDR_PATH; } @@ -1048,11 +1089,11 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, new_state == IB_QPS_RTR && (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || ibqp->qp_type == IB_QPT_UD)) { - context->pri_path.sched_queue = (qp->port - 1) << 6; + context->path.pri_path.sched_queue = (qp->port - 1) << 6; if (is_qp0(dev, qp)) - context->pri_path.sched_queue |= MLX4_IB_DEFAULT_QP0_SCHED_QUEUE; + context->path.pri_path.sched_queue |= MLX4_IB_DEFAULT_QP0_SCHED_QUEUE; else - context->pri_path.sched_queue |= MLX4_IB_DEFAULT_SCHED_QUEUE; + context->path.pri_path.sched_queue |= MLX4_IB_DEFAULT_SCHED_QUEUE; } if (cur_state == IB_QPS_RTS && new_state == IB_QPS_SQD && @@ -1061,6 +1102,17 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, else sqd_event = 0; + if (attr_mask & IB_QP_RCA) { + context->path.pri_path.rss.tbl_sz_base_qpn = + cpu_to_be32(attr->rca.base_qpn | ilog2(attr->rca.num_qpn) << 24); + context->path.pri_path.rss.default_qpn = cpu_to_be32(attr->rca.default_qpn); + context->rca.key.flags_hash_fn = cpu_to_be32(MLX4_RCA_TCP_IPV6 | + MLX4_RCA_IPV6 | + MLX4_RCA_TCP_IPV4 | + MLX4_RCA_IPV4); + memset(context->rca.key.rca_key, 0, sizeof context->rca.key.rca_key); + } + /* * Before passing a kernel QP to the HW, make sure that the * ownership bits of the send queue are set and the SQ @@ -1182,6 +1234,12 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, goto out; } + if (attr_mask & IB_QP_RCA){ + if ((cur_state != IB_QPS_RESET || new_state != IB_QPS_INIT) && + (cur_state != IB_QPS_RTS || new_state != IB_QPS_RTS)) + goto out; + } + err = __mlx4_ib_modify_qp(ibqp, attr, attr_mask, cur_state, new_state); out: @@ -1805,17 +1863,17 @@ int mlx4_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr to_ib_qp_access_flags(be32_to_cpu(context.params2)); if (qp->ibqp.qp_type == IB_QPT_RC || qp->ibqp.qp_type == IB_QPT_UC) { - to_ib_ah_attr(dev->dev, &qp_attr->ah_attr, &context.pri_path); - to_ib_ah_attr(dev->dev, &qp_attr->alt_ah_attr, &context.alt_path); - qp_attr->alt_pkey_index = context.alt_path.pkey_index & 0x7f; + to_ib_ah_attr(dev->dev, &qp_attr->ah_attr, &context.path.pri_path); + to_ib_ah_attr(dev->dev, &qp_attr->alt_ah_attr, &context.path.alt_path); + qp_attr->alt_pkey_index = context.path.alt_path.pkey_index & 0x7f; qp_attr->alt_port_num = qp_attr->alt_ah_attr.port_num; } - qp_attr->pkey_index = context.pri_path.pkey_index & 0x7f; + qp_attr->pkey_index = context.path.pri_path.pkey_index & 0x7f; if (qp_attr->qp_state == IB_QPS_INIT) qp_attr->port_num = qp->port; else - qp_attr->port_num = context.pri_path.sched_queue & 0x40 ? 2 : 1; + qp_attr->port_num = context.path.pri_path.sched_queue & 0x40 ? 2 : 1; /* qp_attr->en_sqd_async_notify is only applicable in modify qp */ qp_attr->sq_draining = mlx4_state == MLX4_QP_STATE_SQ_DRAINING; @@ -1826,10 +1884,10 @@ int mlx4_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr 1 << ((be32_to_cpu(context.params2) >> 21) & 0x7); qp_attr->min_rnr_timer = (be32_to_cpu(context.rnr_nextrecvpsn) >> 24) & 0x1f; - qp_attr->timeout = context.pri_path.ackto >> 3; + qp_attr->timeout = context.path.pri_path.ackto >> 3; qp_attr->retry_cnt = (be32_to_cpu(context.params1) >> 16) & 0x7; qp_attr->rnr_retry = (be32_to_cpu(context.params1) >> 13) & 0x7; - qp_attr->alt_timeout = context.alt_path.ackto >> 3; + qp_attr->alt_timeout = context.path.alt_path.ackto >> 3; done: qp_attr->cur_qp_state = qp_attr->qp_state; diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 1bb2ba4..333afce 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -95,11 +95,22 @@ enum { MLX4_QP_BIT_RIC = 1 << 4, }; +struct mlx4_net_path { + u16 reserved; + __be16 dmac_hi; + __be32 dmac_lo; +}; + +struct mlx4_rss_path { + __be32 tbl_sz_base_qpn; + __be32 default_qpn; +}; + struct mlx4_qp_path { u8 fl; u8 reserved1[2]; u8 pkey_index; - u8 reserved2; + u8 counter_index; u8 grh_mylmc; __be16 rlid; u8 ackto; @@ -111,8 +122,33 @@ struct mlx4_qp_path { u8 sched_queue; u8 snooper_flags; u8 reserved3[2]; - u8 counter_index; - u8 reserved4[7]; + union { + struct mlx4_net_path net; + struct mlx4_rss_path rss; + }; +}; + +struct mlx4_addr_path { + struct mlx4_qp_path pri_path; + struct mlx4_qp_path alt_path; +}; + +enum { + MLX4_RCA_TCP_IPV6 = 1 << 2, + MLX4_RCA_IPV6 = 1 << 3, + MLX4_RCA_TCP_IPV4 = 1 << 4, + MLX4_RCA_IPV4 = 1 << 5, + MLX4_HASH_FN_OFF = 8 +}; + +struct mlx4_rca_key { + __be32 flags_hash_fn; + __be32 rca_key[10]; +}; + +struct mlx4_rca { + struct mlx4_qp_path pri_path; + struct mlx4_rca_key key; }; struct mlx4_qp_context { @@ -125,8 +161,10 @@ struct mlx4_qp_context { __be32 usr_page; __be32 local_qpn; __be32 remote_qpn; - struct mlx4_qp_path pri_path; - struct mlx4_qp_path alt_path; + union { + struct mlx4_addr_path path; + struct mlx4_rca rca; + }; __be32 params1; u32 reserved1; __be32 next_send_psn; -- 1.5.6 From eli at mellanox.co.il Tue Jul 15 09:14:34 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 15 Jul 2008 19:14:34 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: Add Receive Core Affinity (RCA) support Message-ID: <20080715161434.GA8720@mtls03> The patch adds RCA support for datagram mode. It creates a number of additional receive rings, equal to the number of CPU cores, and the counts on the hardware to distribute the received packets based on their IP address and port number; packets that are neither TCP nor UDP, are routed to the regular UD QP. It does some arrangements in the receive related data structs so as to unify processing of received packets to the regular UD QP or to the any of the RCA rings. The receive flow uses napi to provide context for simultaneous handling of received packets. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 35 ++- drivers/infiniband/ulp/ipoib/ipoib_ethtool.c | 63 ++++- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 202 +++++++++++--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 89 +++++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 379 +++++++++++++++++++++++++- 5 files changed, 682 insertions(+), 86 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index b0ffc9a..bb6534d 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -105,6 +105,8 @@ enum { MAX_SEND_CQE = 16, IPOIB_CM_COPYBREAK = 256, + + MAX_RCA_QPS = 8, }; #define IPOIB_OP_RECV (1ul << 31) @@ -267,6 +269,26 @@ struct ipoib_lro { struct net_lro_desc lro_desc[IPOIB_MAX_LRO_DESCRIPTORS]; }; +struct rca_ring_ctx { + struct ib_qp *qp; + struct ib_cq *cq; + struct ib_wc ibwc[IPOIB_NUM_WC]; + struct napi_struct napi; + struct ipoib_rx_buf *rx_ring; + struct ib_recv_wr rx_wr; + struct ib_sge rx_sge[IPOIB_UD_RX_SG]; + struct ipoib_lro lro; + unsigned long rx_packets; + unsigned long rx_dropped; + unsigned long rx_bytes; +}; + +struct ipoib_rca { + struct ib_qp *rca_qp; + int nrings; + struct rca_ring_ctx ring_arr[0]; +}; + /* * Device private locking: tx_lock protects members used in TX fast * path (and we use LLTX so upper layers don't do extra locking). @@ -317,7 +339,7 @@ struct ipoib_dev_priv { unsigned int mcast_mtu; unsigned int max_ib_mtu; - struct ipoib_rx_buf *rx_ring; + struct rca_ring_ctx rx_ring; spinlock_t tx_lock; struct ipoib_tx_buf *tx_ring; @@ -328,11 +350,6 @@ struct ipoib_dev_priv { unsigned tx_outstanding; struct ib_wc send_wc[MAX_SEND_CQE]; - struct ib_recv_wr rx_wr; - struct ib_sge rx_sge[IPOIB_UD_RX_SG]; - - struct ib_wc ibwc[IPOIB_NUM_WC]; - struct list_head dead_ahs; struct ib_event_handler event_handler; @@ -354,7 +371,7 @@ struct ipoib_dev_priv { struct ipoib_ethtool_st ethtool; struct timer_list poll_timer; - struct ipoib_lro lro; + struct ipoib_rca *rca; }; struct ipoib_ah { @@ -427,6 +444,9 @@ extern struct workqueue_struct *ipoib_workqueue; int ipoib_poll(struct napi_struct *napi, int budget); void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr); void ipoib_send_comp_handler(struct ib_cq *cq, void *dev_ptr); +void ipoib_rca_rx_handler(struct ib_cq *cq, void *ptr); +int rca_ring_poll(struct napi_struct *napi, int budget); +void ipoib_rca_qps_to_err(struct net_device *dev); struct ipoib_ah *ipoib_create_ah(struct net_device *dev, struct ib_pd *pd, struct ib_ah_attr *attr); @@ -495,6 +515,7 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, int ipoib_init_qp(struct net_device *dev); int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca); void ipoib_transport_dev_cleanup(struct net_device *dev); +int ipoib_rca_init(struct net_device *dev); void ipoib_event(struct ib_event_handler *handler, struct ib_event *record); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c index 66af5c1..2c64825 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c @@ -60,6 +60,8 @@ static int ipoib_set_coalesce(struct net_device *dev, { struct ipoib_dev_priv *priv = netdev_priv(dev); int ret; + int i; + struct rca_ring_ctx *ring; /* * Since IPoIB uses a single CQ for both rx and tx, we assume @@ -78,6 +80,19 @@ static int ipoib_set_coalesce(struct net_device *dev, return ret; } + if (priv->rca) { + for (i = 0; i < priv->rca->nrings; ++i) { + ring = &priv->rca->ring_arr[i]; + ret = ib_modify_cq(ring->cq, coal->rx_max_coalesced_frames, + coal->rx_coalesce_usecs); + if (ret && ret != -ENOSYS) { + ipoib_warn(priv, "failed modifying CQ for ring %d (%d)\n", i, ret); + return ret; + } + + } + } + coal->tx_coalesce_usecs = coal->rx_coalesce_usecs; coal->tx_max_coalesced_frames = coal->rx_max_coalesced_frames; priv->ethtool.coalesce_usecs = coal->rx_coalesce_usecs; @@ -110,21 +125,57 @@ static int ipoib_get_sset_count(struct net_device *dev, int sset) } } +static u64 get_ring_aggr(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + u64 result; + int i; + + result = priv->rx_ring.lro.lro_mgr.stats.aggregated; + + if (!priv->rca) + return result; + + for (i = 0; i < priv->rca->nrings; ++i) + result += priv->rca->ring_arr[i].lro.lro_mgr.stats.aggregated; + + return result; +} + +static u64 get_ring_flushed(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + u64 result; + int i; + + result = priv->rx_ring.lro.lro_mgr.stats.flushed; + + if (!priv->rca) + return result; + + for (i = 0; i < priv->rca->nrings; ++i) + result += priv->rca->ring_arr[i].lro.lro_mgr.stats.flushed; + + return result; +} + static void ipoib_get_ethtool_stats(struct net_device *dev, struct ethtool_stats *stats, uint64_t *data) { struct ipoib_dev_priv *priv = netdev_priv(dev); int index = 0; + u64 aggr = get_ring_aggr(dev); + u64 flushed = get_ring_flushed(dev); /* Get LRO statistics */ - data[index++] = priv->lro.lro_mgr.stats.aggregated; - data[index++] = priv->lro.lro_mgr.stats.flushed; - if (priv->lro.lro_mgr.stats.flushed) - data[index++] = priv->lro.lro_mgr.stats.aggregated / - priv->lro.lro_mgr.stats.flushed; + data[index++] = aggr; + data[index++] = flushed; + if (flushed) + data[index++] = aggr / flushed; else data[index++] = 0; - data[index++] = priv->lro.lro_mgr.stats.no_desc; + + data[index++] = priv->rx_ring.lro.lro_mgr.stats.no_desc; } static const struct ethtool_ops ipoib_ethtool_ops = { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 66cafa2..e8f686c 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -125,29 +125,32 @@ static void ipoib_ud_skb_put_frags(struct ipoib_dev_priv *priv, } -static int ipoib_ib_post_receive(struct net_device *dev, int id) +static int ipoib_ib_post_receive(struct net_device *dev, + struct rca_ring_ctx *ring, int id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_recv_wr *bad_wr; int ret; - priv->rx_wr.wr_id = id | IPOIB_OP_RECV; - priv->rx_sge[0].addr = priv->rx_ring[id].mapping[0]; - priv->rx_sge[1].addr = priv->rx_ring[id].mapping[1]; + ring->rx_wr.wr_id = id | IPOIB_OP_RECV; + ring->rx_sge[0].addr = ring->rx_ring[id].mapping[0]; + ring->rx_sge[1].addr = ring->rx_ring[id].mapping[1]; - ret = ib_post_recv(priv->qp, &priv->rx_wr, &bad_wr); + ret = ib_post_recv(ring->qp, &ring->rx_wr, &bad_wr); if (unlikely(ret)) { - ipoib_warn(priv, "receive failed for buf %d (%d)\n", id, ret); - ipoib_ud_dma_unmap_rx(priv, priv->rx_ring[id].mapping); - dev_kfree_skb_any(priv->rx_ring[id].skb); - priv->rx_ring[id].skb = NULL; + ipoib_warn(priv, "%s failed for buf %d (%d)\n", __func__, id, ret); + ipoib_ud_dma_unmap_rx(priv, ring->rx_ring[id].mapping); + dev_kfree_skb_any(ring->rx_ring[id].skb); + ring->rx_ring[id].skb = NULL; } return ret; } -static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) +static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, + struct ipoib_rx_buf *rx_ring, + int id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct sk_buff *skb; @@ -170,7 +173,7 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) */ skb_reserve(skb, 4); - mapping = priv->rx_ring[id].mapping; + mapping = rx_ring[id].mapping; mapping[0] = ib_dma_map_single(priv->ca, skb->data, buf_size, DMA_FROM_DEVICE); if (unlikely(ib_dma_mapping_error(priv->ca, mapping[0]))) @@ -188,7 +191,7 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) goto partial_error; } - priv->rx_ring[id].skb = skb; + rx_ring[id].skb = skb; return skb; partial_error: @@ -204,11 +207,31 @@ static int ipoib_ib_post_receives(struct net_device *dev) int i; for (i = 0; i < ipoib_recvq_size; ++i) { - if (!ipoib_alloc_rx_skb(dev, i)) { + if (!ipoib_alloc_rx_skb(dev, priv->rx_ring.rx_ring, i)) { ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); return -ENOMEM; } - if (ipoib_ib_post_receive(dev, i)) { + if (ipoib_ib_post_receive(dev, &priv->rx_ring, i)) { + ipoib_warn(priv, "%s failed for buf %d\n", __func__, i); + return -EIO; + } + } + + return 0; +} + +static int ipoib_post_ring_receives(struct net_device *dev, + struct rca_ring_ctx *ring) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + for (i = 0; i < ipoib_recvq_size; ++i) { + if (!ipoib_alloc_rx_skb(dev, ring->rx_ring, i)) { + ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); + return -ENOMEM; + } + if (ipoib_ib_post_receive(dev, ring, i)) { ipoib_warn(priv, "ipoib_ib_post_receive failed for buf %d\n", i); return -EIO; } @@ -217,7 +240,24 @@ static int ipoib_ib_post_receives(struct net_device *dev) return 0; } -static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) +static int ipoib_rca_post_receives(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int err; + int i; + + for (i = 0; i < priv->rca->nrings; ++i) { + err = ipoib_post_ring_receives(dev, &priv->rca->ring_arr[i]); + if (err) + return err; + } + + return 0; +} + +static void ipoib_ib_handle_rx_wc(struct net_device *dev, + struct rca_ring_ctx *ring, + struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id & ~IPOIB_OP_RECV; @@ -233,16 +273,16 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) return; } - skb = priv->rx_ring[wr_id].skb; + skb = ring->rx_ring[wr_id].skb; if (unlikely(wc->status != IB_WC_SUCCESS)) { if (wc->status != IB_WC_WR_FLUSH_ERR) ipoib_warn(priv, "failed recv event " "(status=%d, wrid=%d vend_err %x)\n", wc->status, wr_id, wc->vendor_err); - ipoib_ud_dma_unmap_rx(priv, priv->rx_ring[wr_id].mapping); + ipoib_ud_dma_unmap_rx(priv, ring->rx_ring[wr_id].mapping); dev_kfree_skb_any(skb); - priv->rx_ring[wr_id].skb = NULL; + ring->rx_ring[wr_id].skb = NULL; return; } @@ -253,15 +293,15 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) if (wc->slid == priv->local_lid && wc->src_qp == priv->qp->qp_num) goto repost; - memcpy(mapping, priv->rx_ring[wr_id].mapping, + memcpy(mapping, ring->rx_ring[wr_id].mapping, IPOIB_UD_RX_SG * sizeof *mapping); /* * If we can't allocate a new RX buffer, dump * this packet and reuse the old buffer. */ - if (unlikely(!ipoib_alloc_rx_skb(dev, wr_id))) { - ++dev->stats.rx_dropped; + if (unlikely(!ipoib_alloc_rx_skb(dev, ring->rx_ring, wr_id))) { + ++ring->rx_dropped; goto repost; } @@ -278,8 +318,8 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb_pull(skb, IPOIB_ENCAP_LEN); dev->last_rx = jiffies; - ++dev->stats.rx_packets; - dev->stats.rx_bytes += skb->len; + ++ring->rx_packets; + ring->rx_bytes += skb->len; skb->dev = dev; /* XXX get correct PACKET_ type here */ @@ -289,12 +329,12 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb->ip_summed = CHECKSUM_UNNECESSARY; if (dev->features & NETIF_F_LRO) - lro_receive_skb(&priv->lro.lro_mgr, skb, NULL); + lro_receive_skb(&ring->lro.lro_mgr, skb, NULL); else netif_receive_skb(skb); repost: - if (unlikely(ipoib_ib_post_receive(dev, wr_id))) + if (unlikely(ipoib_ib_post_receive(dev, ring, wr_id))) ipoib_warn(priv, "ipoib_ib_post_receive failed " "for buf %d\n", wr_id); } @@ -408,10 +448,10 @@ static int poll_tx(struct ipoib_dev_priv *priv) return n == MAX_SEND_CQE; } -int ipoib_poll(struct napi_struct *napi, int budget) +static int rx_poll(struct napi_struct *napi, int budget, struct rca_ring_ctx *ring) { - struct ipoib_dev_priv *priv = container_of(napi, struct ipoib_dev_priv, napi); - struct net_device *dev = priv->dev; + struct net_device *dev = napi->dev; + struct ipoib_dev_priv *priv = netdev_priv(dev); int done; int t; int n, i; @@ -423,17 +463,17 @@ poll_more: int max = (budget - done); t = min(IPOIB_NUM_WC, max); - n = ib_poll_cq(priv->recv_cq, t, priv->ibwc); + n = ib_poll_cq(ring->cq, t, ring->ibwc); for (i = 0; i < n; i++) { - struct ib_wc *wc = priv->ibwc + i; + struct ib_wc *wc = ring->ibwc + i; if (wc->wr_id & IPOIB_OP_RECV) { ++done; if (wc->wr_id & IPOIB_OP_CM) ipoib_cm_handle_rx_wc(dev, wc); else - ipoib_ib_handle_rx_wc(dev, wc); + ipoib_ib_handle_rx_wc(dev, ring, wc); } else ipoib_cm_handle_tx_wc(priv->dev, wc); } @@ -444,10 +484,10 @@ poll_more: if (done < budget) { if (dev->features & NETIF_F_LRO) - lro_flush_all(&priv->lro.lro_mgr); + lro_flush_all(&ring->lro.lro_mgr); netif_rx_complete(dev, napi); - if (unlikely(ib_req_notify_cq(priv->recv_cq, + if (unlikely(ib_req_notify_cq(ring->cq, IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)) && netif_rx_reschedule(dev, napi)) @@ -457,6 +497,28 @@ poll_more: return done; } +int rca_ring_poll(struct napi_struct *napi, int budget) +{ + struct rca_ring_ctx *ring = container_of(napi, struct rca_ring_ctx, napi); + + return rx_poll(napi, budget, ring); +} + +int ipoib_poll(struct napi_struct *napi, int budget) +{ + struct ipoib_dev_priv *priv = netdev_priv(napi->dev); + + return rx_poll(napi, budget, &priv->rx_ring); +} + +void ipoib_rca_rx_handler(struct ib_cq *cq, void *ptr) +{ + struct rca_ring_ctx *ring = ptr; + struct net_device *dev = ring->napi.dev; + + netif_rx_schedule(dev, &ring->napi); +} + void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) { struct net_device *dev = dev_ptr; @@ -677,6 +739,14 @@ int ipoib_ib_dev_open(struct net_device *dev) return -1; } + if (priv->rca) { + ret = ipoib_rca_init(dev); + if (!ret) + ipoib_rca_post_receives(dev); + else + ipoib_warn(priv, "ipoib_rca_init returned %d\n", ret); + } + clear_bit(IPOIB_STOP_REAPER, &priv->flags); queue_delayed_work(ipoib_workqueue, &priv->ah_reap_task, round_jiffies_relative(HZ)); @@ -748,12 +818,22 @@ static int recvs_pending(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); int pending = 0; - int i; + int i, j; for (i = 0; i < ipoib_recvq_size; ++i) - if (priv->rx_ring[i].skb) + if (priv->rx_ring.rx_ring[i].skb) ++pending; + if (!priv->rca) + goto out; + + for (j = 0; j < priv->rca->nrings; ++j) { + for (i = 0; i < ipoib_recvq_size; ++i) + if (priv->rca->ring_arr[j].rx_ring[i].skb) + ++pending; + } + +out: return pending; } @@ -762,23 +842,23 @@ void ipoib_drain_cq(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int i, n; do { - n = ib_poll_cq(priv->recv_cq, IPOIB_NUM_WC, priv->ibwc); + n = ib_poll_cq(priv->recv_cq, IPOIB_NUM_WC, priv->rx_ring.ibwc); for (i = 0; i < n; ++i) { /* * Convert any successful completions to flush * errors to avoid passing packets up the * stack after bringing the device down. */ - if (priv->ibwc[i].status == IB_WC_SUCCESS) - priv->ibwc[i].status = IB_WC_WR_FLUSH_ERR; + if (priv->rx_ring.ibwc[i].status == IB_WC_SUCCESS) + priv->rx_ring.ibwc[i].status = IB_WC_WR_FLUSH_ERR; - if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) { - if (priv->ibwc[i].wr_id & IPOIB_OP_CM) - ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); + if (priv->rx_ring.ibwc[i].wr_id & IPOIB_OP_RECV) { + if (priv->rx_ring.ibwc[i].wr_id & IPOIB_OP_CM) + ipoib_cm_handle_rx_wc(dev, priv->rx_ring.ibwc + i); else - ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); + ipoib_ib_handle_rx_wc(dev, &priv->rx_ring, priv->rx_ring.ibwc + i); } else - ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); + ipoib_cm_handle_tx_wc(dev, priv->rx_ring.ibwc + i); } } while (n == IPOIB_NUM_WC); @@ -786,6 +866,34 @@ void ipoib_drain_cq(struct net_device *dev) ; /* nothing */ } +void ipoib_drain_rca_rx_wrs(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct rca_ring_ctx *ring; + int i, n, k; + + if (!priv->rca) + return; + + for (k = 0; k < priv->rca->nrings; ++k) { + ring = &priv->rca->ring_arr[k]; + do { + n = ib_poll_cq(ring->cq, IPOIB_NUM_WC, ring->ibwc); + for (i = 0; i < n; ++i) { + /* + * Convert any successful completions to flush + * errors to avoid passing packets up the + * stack after bringing the device down. + */ + if (ring->ibwc[i].status == IB_WC_SUCCESS) + ring->ibwc[i].status = IB_WC_WR_FLUSH_ERR; + + ipoib_ib_handle_rx_wc(dev, ring, ring->ibwc + i); + } + } while (n == IPOIB_NUM_WC); + } +} + int ipoib_ib_dev_stop(struct net_device *dev, int flush) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -806,6 +914,8 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) if (ib_modify_qp(priv->qp, &qp_attr, IB_QP_STATE)) ipoib_warn(priv, "Failed to modify QP to ERROR state\n"); + ipoib_rca_qps_to_err(dev); + /* Wait for all sends and receives to complete */ begin = jiffies; @@ -830,11 +940,11 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) for (i = 0; i < ipoib_recvq_size; ++i) { struct ipoib_rx_buf *rx_req; - rx_req = &priv->rx_ring[i]; + rx_req = &priv->rx_ring.rx_ring[i]; if (!rx_req->skb) continue; ipoib_ud_dma_unmap_rx(priv, - priv->rx_ring[i].mapping); + priv->rx_ring.rx_ring[i].mapping); dev_kfree_skb_any(rx_req->skb); rx_req->skb = NULL; } @@ -843,6 +953,8 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) } ipoib_drain_cq(dev); + ipoib_drain_rca_rx_wrs(dev); + msleep(1); } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 8be9ea0..796cf86 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -100,6 +100,31 @@ static struct ib_client ipoib_client = { .remove = ipoib_remove_one }; + +static void enable_rca_napi(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + if (!priv->rca) + return; + + for (i = 0; i < priv->rca->nrings; ++i) + napi_enable(&priv->rca->ring_arr[i].napi); +} + +static void disable_rca_napi(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + if (!priv->rca) + return; + + for (i = 0; i < priv->rca->nrings; ++i) + napi_disable(&priv->rca->ring_arr[i].napi); +} + int ipoib_open(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -107,6 +132,7 @@ int ipoib_open(struct net_device *dev) ipoib_dbg(priv, "bringing up interface\n"); napi_enable(&priv->napi); + enable_rca_napi(dev); set_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags); if (ipoib_pkey_dev_delay_open(dev)) @@ -114,12 +140,14 @@ int ipoib_open(struct net_device *dev) if (ipoib_ib_dev_open(dev)) { napi_disable(&priv->napi); + disable_rca_napi(dev); return -EINVAL; } if (ipoib_ib_dev_up(dev)) { ipoib_ib_dev_stop(dev, 1); napi_disable(&priv->napi); + enable_rca_napi(dev); return -EINVAL; } @@ -153,6 +181,7 @@ static int ipoib_stop(struct net_device *dev) clear_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags); napi_disable(&priv->napi); + disable_rca_napi(dev); netif_stop_queue(dev); @@ -921,9 +950,9 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) struct ipoib_dev_priv *priv = netdev_priv(dev); /* Allocate RX/TX "rings" to hold queued skbs */ - priv->rx_ring = kzalloc(ipoib_recvq_size * sizeof *priv->rx_ring, + priv->rx_ring.rx_ring = kzalloc(ipoib_recvq_size * sizeof *priv->rx_ring.rx_ring, GFP_KERNEL); - if (!priv->rx_ring) { + if (!priv->rx_ring.rx_ring) { printk(KERN_WARNING "%s: failed to allocate RX ring (%d entries)\n", ca->name, ipoib_recvq_size); goto out; @@ -948,7 +977,7 @@ out_tx_ring_cleanup: vfree(priv->tx_ring); out_rx_ring_cleanup: - kfree(priv->rx_ring); + kfree(priv->rx_ring.rx_ring); out: return -ENOMEM; @@ -969,10 +998,10 @@ void ipoib_dev_cleanup(struct net_device *dev) ipoib_ib_dev_cleanup(dev); - kfree(priv->rx_ring); + kfree(priv->rx_ring.rx_ring); vfree(priv->tx_ring); - priv->rx_ring = NULL; + priv->rx_ring.rx_ring = NULL; priv->tx_ring = NULL; } @@ -1017,15 +1046,33 @@ static int get_skb_hdr(struct sk_buff *skb, void **iphdr, return 0; } -static void ipoib_lro_setup(struct ipoib_dev_priv *priv) +static void ipoib_lro_setup(struct ipoib_lro *lro, struct net_device *dev) +{ + lro->lro_mgr.max_aggr = lro_max_aggr; + lro->lro_mgr.max_desc = IPOIB_MAX_LRO_DESCRIPTORS; + lro->lro_mgr.lro_arr = lro->lro_desc; + lro->lro_mgr.get_skb_header = get_skb_hdr; + lro->lro_mgr.features = LRO_F_NAPI; + lro->lro_mgr.dev = dev; + lro->lro_mgr.ip_summed_aggr = CHECKSUM_UNNECESSARY; +} + +struct net_device_stats* ipoib_get_stats(struct net_device *dev) { - priv->lro.lro_mgr.max_aggr = lro_max_aggr; - priv->lro.lro_mgr.max_desc = IPOIB_MAX_LRO_DESCRIPTORS; - priv->lro.lro_mgr.lro_arr = priv->lro.lro_desc; - priv->lro.lro_mgr.get_skb_header = get_skb_hdr; - priv->lro.lro_mgr.features = LRO_F_NAPI; - priv->lro.lro_mgr.dev = priv->dev; - priv->lro.lro_mgr.ip_summed_aggr = CHECKSUM_UNNECESSARY; + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + dev->stats.rx_packets = priv->rx_ring.rx_packets; + dev->stats.rx_dropped = priv->rx_ring.rx_dropped; + dev->stats.rx_bytes = priv->rx_ring.rx_bytes; + if (priv->rca) { + for (i = 0; i < priv->rca->nrings; ++i) { + dev->stats.rx_packets += priv->rca->ring_arr[i].rx_packets; + dev->stats.rx_dropped += priv->rca->ring_arr[i].rx_dropped; + dev->stats.rx_bytes += priv->rca->ring_arr[i].rx_bytes; + } + } + return &dev->stats; } static void ipoib_setup(struct net_device *dev) @@ -1040,6 +1087,7 @@ static void ipoib_setup(struct net_device *dev) dev->header_ops = &ipoib_header_ops; dev->set_multicast_list = ipoib_set_mcast_list; dev->neigh_setup = ipoib_neigh_setup_dev; + dev->get_stats = ipoib_get_stats; ipoib_set_ethtool_ops(dev); @@ -1067,7 +1115,7 @@ static void ipoib_setup(struct net_device *dev) priv->dev = dev; - ipoib_lro_setup(priv); + ipoib_lro_setup(&priv->rx_ring.lro, dev); spin_lock_init(&priv->lock); spin_lock_init(&priv->tx_lock); @@ -1190,6 +1238,16 @@ int ipoib_add_pkey_attr(struct net_device *dev) return device_create_file(&dev->dev, &dev_attr_pkey); } +static void ipoib_register_rings(struct ipoib_dev_priv *priv) +{ + int i; + + for (i = 0; i < priv->rca->nrings; ++i) { + netif_napi_add(priv->dev, &priv->rca->ring_arr[i].napi, rca_ring_poll, 100); + ipoib_lro_setup(&priv->rca->ring_arr[i].lro, priv->dev); + } +} + static struct net_device *ipoib_add_port(const char *format, struct ib_device *hca, u8 port) { @@ -1273,6 +1331,9 @@ static struct net_device *ipoib_add_port(const char *format, goto device_init_failed; } + if (priv->rca) + ipoib_register_rings(priv); + INIT_IB_EVENT_HANDLER(&priv->event_handler, priv->ca, ipoib_event); result = ib_register_event_handler(&priv->event_handler); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 6832511..3ded8f5 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -33,6 +33,35 @@ #include "ipoib.h" +static int rca; +module_param(rca, bool, 0444); +MODULE_PARM_DESC(rca, "Enable RCA (Receive Core Affinity)"); + +static int set_rca_qkey(struct ipoib_dev_priv *priv) +{ + struct ib_qp_attr *qp_attr; + int ret = -ENOMEM; + int i; + + if (!priv->rca) + return 0; + + qp_attr = kmalloc(sizeof *qp_attr, GFP_KERNEL); + if (!qp_attr) + return -ENOMEM; + + qp_attr->qkey = priv->qkey; + for (i = 0; i < priv->rca->nrings; ++i) { + ret = ib_modify_qp(priv->rca->ring_arr[i].qp, qp_attr, IB_QP_QKEY); + if (ret) + goto out; + } + +out: + kfree(qp_attr); + return ret; +} + int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -60,6 +89,12 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int ipoib_warn(priv, "failed to modify QP, ret = %d\n", ret); goto out; } + + ret = set_rca_qkey(priv); + if (ret) { + ipoib_warn(priv, "failed to modify RCA QPs qkey, ret = %d\n", ret); + goto out; + } } /* attach QP to multicast group */ @@ -126,6 +161,309 @@ out_fail: return ret; } +static void qp2reset(struct ipoib_dev_priv *priv, struct ib_qp *qp) +{ + struct ib_qp_attr qp_attr; + + qp_attr.qp_state = IB_QPS_RESET; + if (ib_modify_qp(qp, &qp_attr, IB_QP_STATE)) + ipoib_warn(priv, "Failed to modify QP to RESET state\n"); +} + +static void qp2err(struct ipoib_dev_priv *priv, struct ib_qp *qp) +{ + struct ib_qp_attr qp_attr; + + qp_attr.qp_state = IB_QPS_ERR; + if (ib_modify_qp(qp, &qp_attr, IB_QP_STATE)) + ipoib_warn(priv, "Failed to modify QP to ERROR state\n"); +} + +static int modify_to_rts(struct ipoib_dev_priv *priv, struct ib_qp *qp) // ?? need to modify to rtr +{ + int ret; + struct ib_qp_attr qp_attr; + int attr_mask; + + qp_attr.qp_state = IB_QPS_INIT; + qp_attr.qkey = 0; + qp_attr.port_num = priv->port; + qp_attr.pkey_index = priv->pkey_index; + attr_mask = + IB_QP_QKEY | + IB_QP_PORT | + IB_QP_PKEY_INDEX | + IB_QP_STATE; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to init, ret = %d\n", ret); + goto out_fail; + } + + qp_attr.qp_state = IB_QPS_RTR; + /* Can't set this in a INIT->RTR transition */ + attr_mask &= ~IB_QP_PORT; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to RTR, ret = %d\n", ret); + goto out_fail; + } + + qp_attr.qp_state = IB_QPS_RTS; + qp_attr.sq_psn = 0; + attr_mask |= IB_QP_SQ_PSN; + attr_mask &= ~IB_QP_PKEY_INDEX; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to RTS, ret = %d\n", ret); + goto out_fail; + } + + return 0; + +out_fail: + qp2reset(priv, qp); + return ret; +} + +static int prepare_rca_qp(struct ipoib_dev_priv *priv, struct ib_qp *qp) +{ + int ret; + struct ib_qp_attr qp_attr; + int attr_mask; + + qp_attr.qp_state = IB_QPS_INIT; + qp_attr.qkey = 0; + qp_attr.port_num = priv->port; + qp_attr.pkey_index = priv->pkey_index; + qp_attr.rca.base_qpn = priv->rca->ring_arr[0].qp->qp_num; + qp_attr.rca.num_qpn = priv->rca->nrings; + qp_attr.rca.default_qpn = priv->qp->qp_num; + + attr_mask = + IB_QP_QKEY | + IB_QP_PORT | + IB_QP_PKEY_INDEX | + IB_QP_STATE | + IB_QP_RCA; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to init, ret = %d\n", ret); + goto out_fail; + } + + qp_attr.qp_state = IB_QPS_RTR; + /* Can't set this in a INIT->RTR transition */ + attr_mask &= ~(IB_QP_PORT | IB_QP_RCA); + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to RTR, ret = %d\n", ret); + goto out_fail; + } + + return 0; + +out_fail: + qp2reset(priv, qp); + return ret; +} + +int ipoib_rca_init(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int err; + int i; + + for (i = 0; i < priv->rca->nrings; ++i) { + err = modify_to_rts(priv, priv->rca->ring_arr[i].qp); + if (err) { + ipoib_warn(priv, "ring[%d] failed modify to RTS\n", i); + goto exit_err; + } + } + + printk(KERN_WARNING "%s-%d\n", __func__, __LINE__); + err = prepare_rca_qp(priv, priv->rca->rca_qp); + if (err) + goto exit_err; + + return 0; + +exit_err: + for (--i; i >= 0; --i) + qp2reset(priv, priv->rca->ring_arr[i].qp); + + return err; +} + +void ipoib_rca_qps_to_err(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + if (!priv->rca) + return; + + for (i = 0; i < priv->rca->nrings; ++i) + qp2err(priv, priv->rca->ring_arr[i].qp); +} + +static void destroy_rca_resources(struct ipoib_dev_priv *priv) +{ + int i; + int err; + + for (i = 0; i < priv->rca->nrings; ++i) { + if (priv->rca->ring_arr[i].qp) { + err = ib_destroy_qp(priv->rca->ring_arr[i].qp); + if (err) + printk(KERN_WARNING "%s: failed to destroy qp[%d], err = %d\n", + __func__, i, err); + } + + if (priv->rca->ring_arr[i].cq) { + err = ib_destroy_cq(priv->rca->ring_arr[i].cq); + if (err) + printk(KERN_WARNING "%s: failed to destroy cq[%d], err = %d\n", + __func__, i, err); + } + + kfree(priv->rca->ring_arr[i].rx_ring); + } + + if (priv->rca->rca_qp) + if (ib_destroy_qp(priv->rca->rca_qp)) + printk(KERN_WARNING "failed to destroy rca qp\n"); + + kfree(priv->rca); + priv->rca = NULL; +} + +static void qpi_rx_event_handler(struct ib_event *event, void *ctx) +{ + struct ipoib_dev_priv *priv = ctx; + + ipoib_warn(priv, "got event %d on RCA QP. QPN = 0x%x\n", event->event, + event->element.qp->qp_num); +} + +static void rca_qp_rx_event_handler(struct ib_event *event, void *ctx) +{ + struct ipoib_dev_priv *priv = ctx; + + ipoib_warn(priv, "RCA QP got event %d. QPN = 0x%x\n", event->event, + event->element.qp->qp_num); +} + +static void init_ring(struct ipoib_dev_priv *priv, struct rca_ring_ctx *ring) +{ + ring->rx_sge[0].lkey = priv->mr->lkey; + if (ipoib_ud_need_sg(priv->max_ib_mtu)) { + ring->rx_sge[0].length = IPOIB_UD_HEAD_SIZE; + ring->rx_sge[1].length = PAGE_SIZE; + ring->rx_sge[1].lkey = priv->mr->lkey; + ring->rx_wr.num_sge = IPOIB_UD_RX_SG; + } else { + ring->rx_sge[0].length = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); + ring->rx_wr.num_sge = 1; + } + ring->rx_wr.next = NULL; + ring->rx_wr.sg_list = ring->rx_sge; +} + +static int create_rca_resources(struct ipoib_dev_priv *priv, struct ib_device *ca) +{ + struct ib_qp_init_attr *attr; + int err; + int i; + int num_comp_vectors = ca->num_comp_vectors; + struct ib_qp **qp_arr; + + priv->rca = kzalloc(sizeof *priv->rca + num_comp_vectors * + sizeof *priv->rca->ring_arr, GFP_KERNEL); + if (!priv->rca) + return -ENOMEM; + + attr = kzalloc(num_comp_vectors * sizeof *attr, GFP_KERNEL); + if (!attr) { + err = -ENOMEM; + goto exit_err; + } + qp_arr = kmalloc(num_comp_vectors * sizeof *qp_arr, GFP_KERNEL); + if (!qp_arr) { + kfree(attr); + err = -ENOMEM; + goto exit_err; + } + + for (i = 0; i < num_comp_vectors; ++i) { + priv->rca->ring_arr[i].rx_ring = kzalloc(sizeof(struct ipoib_rx_buf) * + ipoib_recvq_size, GFP_KERNEL); + if (!priv->rca->ring_arr[i].rx_ring) { + err = -ENOMEM; + goto exit_free; + } + + init_ring(priv, &priv->rca->ring_arr[i]); + + priv->rca->ring_arr[i].cq = ib_create_cq(ca, ipoib_rca_rx_handler, NULL, + &priv->rca->ring_arr[i], ipoib_recvq_size, i); + if (IS_ERR(priv->rca->ring_arr[i].cq)) { + err = PTR_ERR(priv->rca->ring_arr[i].cq); + priv->rca->ring_arr[i].cq = NULL; + goto exit_free; + } + + err = ib_req_notify_cq(priv->rca->ring_arr[i].cq, IB_CQ_NEXT_COMP); + if (err) + goto exit_free; + + attr[i].event_handler = qpi_rx_event_handler; + attr[i].qp_context = priv; + attr[i].sq_sig_type = IB_SIGNAL_ALL_WR; + attr[i].qp_type = IB_QPT_UD; + attr[i].cap.max_recv_wr = ipoib_recvq_size; + attr[i].cap.max_recv_sge = IPOIB_UD_RX_SG; + attr[i].recv_cq = attr[i].send_cq = priv->rca->ring_arr[i].cq; + } + + err = ib_create_qp_range(priv->pd, attr, num_comp_vectors, + 1 << ilog2(num_comp_vectors), qp_arr); + if (err) + goto exit_free; + + for (i = 0; i < num_comp_vectors; ++i) + priv->rca->ring_arr[i].qp = qp_arr[i]; + + memset(attr, 0, sizeof *attr); + attr[0].event_handler = rca_qp_rx_event_handler; + attr[0].qp_context = priv; + attr[0].qp_type = IB_QPT_UD; + attr[0].recv_cq = attr[0].send_cq = priv->recv_cq; + attr[0].create_flags = IB_QP_CREATE_IPOIB_RCA; + priv->rca->rca_qp = ib_create_qp(priv->pd, attr); + if (IS_ERR(priv->rca->rca_qp)) { + err = PTR_ERR(priv->rca->rca_qp); + priv->rca->rca_qp = NULL; + goto exit_free; + } + + kfree(qp_arr); + kfree(attr); + priv->rca->nrings = num_comp_vectors; + + return 0; + +exit_free: + kfree(qp_arr); + kfree(attr); + +exit_err: + destroy_rca_resources(priv); + + return err; +} + int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -139,7 +477,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_UD }; - + u32 hw_qpn; int ret, size; int i; @@ -199,10 +537,6 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_send_cq; } - priv->dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; - priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; - priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) priv->tx_sge[i].lkey = priv->mr->lkey; @@ -210,18 +544,32 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; - priv->rx_sge[0].lkey = priv->mr->lkey; + priv->rx_ring.rx_sge[0].lkey = priv->mr->lkey; if (ipoib_ud_need_sg(priv->max_ib_mtu)) { - priv->rx_sge[0].length = IPOIB_UD_HEAD_SIZE; - priv->rx_sge[1].length = PAGE_SIZE; - priv->rx_sge[1].lkey = priv->mr->lkey; - priv->rx_wr.num_sge = IPOIB_UD_RX_SG; + priv->rx_ring.rx_sge[0].length = IPOIB_UD_HEAD_SIZE; + priv->rx_ring.rx_sge[1].length = PAGE_SIZE; + priv->rx_ring.rx_sge[1].lkey = priv->mr->lkey; + priv->rx_ring.rx_wr.num_sge = IPOIB_UD_RX_SG; } else { - priv->rx_sge[0].length = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); - priv->rx_wr.num_sge = 1; + priv->rx_ring.rx_sge[0].length = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); + priv->rx_ring.rx_wr.num_sge = 1; + } + priv->rx_ring.rx_wr.next = NULL; + priv->rx_ring.rx_wr.sg_list = priv->rx_ring.rx_sge; + priv->rx_ring.qp = priv->qp; + priv->rx_ring.cq = priv->recv_cq; + + hw_qpn = priv->qp->qp_num; + if (rca && priv->hca_caps & IB_DEVICE_IPOIB_RCA) { + if (create_rca_resources(priv, ca)) + printk(KERN_WARNING "not using RCA for device %s\n", ca->name); + else + hw_qpn = priv->rca->rca_qp->qp_num; } - priv->rx_wr.next = NULL; - priv->rx_wr.sg_list = priv->rx_sge; + + priv->dev->dev_addr[1] = (hw_qpn >> 16) & 0xff; + priv->dev->dev_addr[2] = (hw_qpn >> 8) & 0xff; + priv->dev->dev_addr[3] = hw_qpn & 0xff; return 0; @@ -244,6 +592,9 @@ void ipoib_transport_dev_cleanup(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); + if (priv->rca) + destroy_rca_resources(priv); + if (priv->qp) { if (ib_destroy_qp(priv->qp)) ipoib_warn(priv, "ib_qp_destroy failed\n"); -- 1.5.6 From FENKES at de.ibm.com Tue Jul 15 09:26:22 2008 From: FENKES at de.ibm.com (Joachim Fenkes) Date: Tue, 15 Jul 2008 18:26:22 +0200 Subject: [ofa-general] Re: [ewg] [PATCH] IB/ehca: Backport for RHEL 4.6 In-Reply-To: <487CC82B.1040501@mellanox.co.il> References: <200807151627.44553.fenkes@de.ibm.com> <487CC82B.1040501@mellanox.co.il> Message-ID: Vladimir Sokolovsky wrote on 15.07.2008 17:54:19: > Applied, Thanks! > Should the same be done for 2.6.9_U4 and 2.6.9_U5? I'd rather not; we haven't tested those. Regards, Joachim From sean.hefty at intel.com Tue Jul 15 09:56:39 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 15 Jul 2008 09:56:39 -0700 Subject: [ofa-general] IB verbs client/server programming... In-Reply-To: <2f3bf9a60807150707g4323b997t9914bec19613b5ab@mail.gmail.com> References: <1216054280.15841.28.camel@hermosa.morrealenet><2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com><1216127265.15841.67.camel@hermosa.morrealenet> <2f3bf9a60807150707g4323b997t9914bec19613b5ab@mail.gmail.com> Message-ID: <000001c8e69b$c05849b0$27fc070a@amr.corp.intel.com> >> On a related note... Is it possible to reuse a cma_id for different >> memory regions by appropriately calling ibv_reg_mr() and >> ibv_dereg_mr()? >Sorry, but i don't really know the answer for this. >You should contact Sean Hefty for questions about the cma ... Memory registrations are done through the verbs interfaces, not the rdma_cm. But you can call ibv_reg_mr() multiple times to register different memory regions on the same protection domain. - Sean From sean.hefty at intel.com Tue Jul 15 10:03:27 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 15 Jul 2008 10:03:27 -0700 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <487C816C.7040301@mellanox.co.il> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> <487B8727.20003@opengridcomputing.com> <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> <487C816C.7040301@mellanox.co.il> Message-ID: <000101c8e69c$b35025c0$27fc070a@amr.corp.intel.com> >> - Question about IBCM for OFED v1.4: >> - Any estimate on the following ticket: >> https://bugs.openfabrics.org/show_bug.cgi?id=963 >> - Any estimate on the "ibcm_listen() sometimes failing" issue? (I >> don't think there's a ticket filed against this yet) >> >> >I don't think anything was done here - Sean? I have been working with Jeff and the ompi developers directly on a different thread, but I don't think that ofa general is copied on it. There's a possibility that ompi is colliding with the SDP listen space, and we're exploring that option. - Sean From dotanba at gmail.com Tue Jul 15 11:19:14 2008 From: dotanba at gmail.com (Dotan Barak) Date: Tue, 15 Jul 2008 20:19:14 +0200 Subject: [ofa-general] IB verbs client/server programming... In-Reply-To: <1216137311.15841.124.camel@hermosa.morrealenet> References: <1216054280.15841.28.camel@hermosa.morrealenet> <2f3bf9a60807142339j52c4ff32la49e46439877f728@mail.gmail.com> <1216127265.15841.67.camel@hermosa.morrealenet> <2f3bf9a60807150707g4323b997t9914bec19613b5ab@mail.gmail.com> <1216137311.15841.124.camel@hermosa.morrealenet> Message-ID: <487CEA22.4080308@gmail.com> Peter W. Morreale wrote: > On Tue, 2008-07-15 at 17:07 +0300, Dotan Barak wrote: > >> On Tue, Jul 15, 2008 at 4:07 PM, Peter W. Morreale wrote: >> >>> On Tue, 2008-07-15 at 09:39 +0300, Dotan Barak wrote: >>> >>> >>>>> In other words, what interfaces do I have to look at to implement the >>>>> "blocking" portion equivalent of a recv(2)? >>>>> >>>> Do you want to block until the data will be received? >>>> >>> Yes. Right now, all the examples I see involve a tight polling loop >>> waiting for, and acknowledging the events. >>> >>> >> You need to poll for completion on the receive queue >> (IF you use opcode that will consume WR in the recieve queue at the receiver) >> >> > > I'm sorry, I don't understand this. What do you mean by "If you use > opcode that will consume the work request in the receive queue..." > > What opcodes are you referring to? > > The opcodes: Send, Send with immediate, RDMA Write with immediate consumes WR from the receive QP of the receiver, the other opcodes don't > Are you saying that the receiver does not have to acknowledge the > receipt of the data transfer? (That would be ideal for the data > transfers.) If so, how do I set that up? > This is what RDMA is all about: the ability to write/read data directly to/from the remote process memory. It is out of the scope of Infiniband (but it is in the scope of the application) on how to inform the other side that the operations was executed and its status. > The examples I have seen have the receiver ack'ing events. Can these > be deferred (or ignored even?) I don't really understand what you mean by saying this sentence, can you please try to explain it to me (and write the code lines that does this operation). Maybe you are referring to the code that read completions using events... > > > Thanks, > -PWM > Dotan From rdreier at cisco.com Tue Jul 15 10:56:20 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 10:56:20 -0700 Subject: [ofa-general] Re: [ewg] [PATCH] IB/core: remove an always true condition In-Reply-To: <20080714072201.GA8348@mtls03> (Eli Cohen's message of "Mon, 14 Jul 2008 10:22:01 +0300") References: <20080714072201.GA8348@mtls03> Message-ID: > - if (hdr.command < 0 || > - hdr.command >= ARRAY_SIZE(uverbs_cmd_table) || > + if (hdr.command >= ARRAY_SIZE(uverbs_cmd_table) || I prefer to leave the code in as being more self-documenting and resilient against future changes. The compiler should be able to optimize away the always-true part anyway. From john.russo at qlogic.com Tue Jul 15 11:08:41 2008 From: john.russo at qlogic.com (John Russo) Date: Tue, 15 Jul 2008 13:08:41 -0500 Subject: [ofa-general] Question In-Reply-To: <487CC4B8.5080506@mellanox.co.il> References: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org> <487CC4B8.5080506@mellanox.co.il> Message-ID: <99863D2ED484D449811D97A4C44C9CBD82FAF0@EPEXCH2.qlogic.org> We have found that causes openibd to be started before networking and therefore the NodeDescription, when returned from the SM, does not always contain the hostname of the system when ibhosts is run. Is this a problem for anyone else? -----Original Message----- From: Vladimir Sokolovsky [mailto:vlad at mellanox.co.il] Sent: Tuesday, July 15, 2008 11:40 AM To: John Russo Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] Question John Russo wrote: > I apologize if I missed this when it occurred... > > > > Between 1.3 and 1.3.1 why were the chkconfig values (for RedHat) changed > in the spec file from 15 85 to 05 95 and (for SLES) the requirement for > network to be a required-start removed? > > > > * > ***__________________________*** > **John F. Russo** > **Manager, Engineering** > **QLogic Corporation** > **780 Fifth Avenue, Suite 140** > **King of Prussia****, PA 19406** > **Direct: 610-233-4866** > **Main****: 610-233-4800** > **Fax: 610-233-4777** > **Cell: 610-246-9903** > **Email: *****John.Russo at qlogic.com** > **www.qlogic.com** > This changes the start order of /etc/init.d/network and /etc/init.d/openibd scripts. Now the openibd script start first. And this enables bonding module to be configured correctly (after ib_ipoib module being loaded). Regards, Vladimir From xma at us.ibm.com Tue Jul 15 11:11:25 2008 From: xma at us.ibm.com (Shirley Ma) Date: Tue, 15 Jul 2008 11:11:25 -0700 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support In-Reply-To: <20080715161251.GA16299@mtls03> Message-ID: general-bounces at lists.openfabrics.org wrote on 07/15/2008 09:12:51 AM: > Following this email is a list of 3 patches which implements receive > core affinity for IPoIB. In tests I made on my machine I got the > following results with netperf TCP, 4 streams. > > Without RCA: 730 MB/s > With RCA: 970 MB/s > > The patches require the patches sent by Yevegeni P. to provide support > for allocation of multiple queues. I have sent an RFC a few weeks ago > but it did not receive too much attention, so I implemented on the > original API I sent back then. I would like this to be reviewed by the > community and good comments are welcome. That's a good idea. We talked about this internally. It benefits both UD and CM mode. Does this RCA share same CQ? Or different RCA could use different CQ/EQ per core? Thanks Shirley From rdreier at cisco.com Tue Jul 15 11:25:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 11:25:11 -0700 Subject: [ofa-general] Re: [PATCH] mlx4: Update/add Melanox Technologies copyright lines to mlx4 driver files (for 2.6.27) In-Reply-To: <200807141028.20625.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Mon, 14 Jul 2008 10:28:20 +0300") References: <200807141028.20625.jackm@dev.mellanox.co.il> Message-ID: > diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c > index c75ac94..2906e82 100644 > --- a/drivers/infiniband/hw/mlx4/ah.c > +++ b/drivers/infiniband/hw/mlx4/ah.c > @@ -1,5 +1,6 @@ > /* > * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. > + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. I just took a quick look at the log for this first file in the patch, and I don't see any changes since I originally wrote the file. So I'm not sure this makes sense. Further browsing shows copyright notices added to files that no one from Mellanox touched, or at least didn't touch in the years added. I don't have any problem merging stuff like this, although I don't like to do it too often, but please exercise some care to make sure your changes are actually accurate. From sean.hefty at intel.com Tue Jul 15 11:32:25 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 15 Jul 2008 11:32:25 -0700 Subject: [ofa-general] RE: [PATCH] CMA: Notify ULP when CM exits the TIMEWAIT state. In-Reply-To: <12161313341500-git-send-email-amirv@mellanox.co.il> References: <12161313341500-git-send-email-amirv@mellanox.co.il> Message-ID: <000201c8e6a9$20f8fff0$27fc070a@amr.corp.intel.com> from: Signed-off-by: Amir Vadai Applications that want to re-use their QPs in new connections need to know when the QP has exited the timewait state. Report the timewait event through the rdma_cm. Signed-off-by: Amir Vadai Acked-by: Sean Hefty --- Roland, this obviously isn't critical, but it would be nice to get into 2.6.27 if possible, to include it in the next release of the librdmacm. > drivers/infiniband/core/cma.c | 7 ++++++- > include/rdma/rdma_cm.h | 3 ++- > 2 files changed, 8 insertions(+), 2 deletions(-) > >diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c >index ae11d5c..9fa1371 100644 >--- a/drivers/infiniband/core/cma.c >+++ b/drivers/infiniband/core/cma.c >@@ -914,7 +914,10 @@ static int cma_ib_handler(struct ib_cm_id *cm_id, struct >ib_cm_event *ib_event) > struct rdma_cm_event event; > int ret = 0; > >- if (cma_disable_callback(id_priv, CMA_CONNECT)) >+ if ((ib_event->event != IB_CM_TIMEWAIT_EXIT && >+ cma_disable_callback(id_priv, CMA_CONNECT)) || >+ (ib_event->event == IB_CM_TIMEWAIT_EXIT && >+ cma_disable_callback(id_priv, CMA_DISCONNECT))) > return 0; > > memset(&event, 0, sizeof event); >@@ -950,6 +953,8 @@ static int cma_ib_handler(struct ib_cm_id *cm_id, struct >ib_cm_event *ib_event) > event.event = RDMA_CM_EVENT_DISCONNECTED; > break; > case IB_CM_TIMEWAIT_EXIT: >+ event.event = RDMA_CM_EVENT_TIMWAIT_EXIT; >+ break; > case IB_CM_MRA_RECEIVED: > /* ignore event */ > goto out; >diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h >index 22bb2e7..80170bd 100644 >--- a/include/rdma/rdma_cm.h >+++ b/include/rdma/rdma_cm.h >@@ -57,7 +57,8 @@ enum rdma_cm_event_type { > RDMA_CM_EVENT_DISCONNECTED, > RDMA_CM_EVENT_DEVICE_REMOVAL, > RDMA_CM_EVENT_MULTICAST_JOIN, >- RDMA_CM_EVENT_MULTICAST_ERROR >+ RDMA_CM_EVENT_MULTICAST_ERROR, >+ RDMA_CM_EVENT_TIMWAIT_EXIT > }; > > enum rdma_port_space { >-- >1.5.3 From rdreier at cisco.com Tue Jul 15 11:41:55 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 11:41:55 -0700 Subject: [ofa-general] Re: [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) In-Reply-To: <200807151745.53152.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Tue, 15 Jul 2008 17:45:52 +0300") References: <200807151745.53152.jackm@dev.mellanox.co.il> Message-ID: thanks, applied... a few comments: > Subject: [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) please put the (for 2.6.27) inside the [] with "PATCH" so that it gets automatically stripped. As it is I have to edit this out by hand. > Content-Transfer-Encoding: quoted-printable argh... the patch ends up looking like: > =2D-- a/drivers/net/mlx4/fw.c > +++ b/drivers/net/mlx4/fw.c > @@ -46,6 +46,10 @@ enum { > extern void __buggy_use_of_MLX4_GET(void); > extern void __buggy_use_of_MLX4_PUT(void); > =20 and I have to fix that by hand too. > From 01d8c13b6dd314b239a5b0b76e57320aeaa5c338 Mon Sep 17 00:00:00 2001 please leave that header out too... I have to edit *that* out by hand as well. > +static int mlx4_core_enable_qos; > +module_param_named(enable_qos, mlx4_core_enable_qos, int, 0444); given that it's static, why not just static int enable_qos; module_param(enable_qos, bool, 0444); (and make it a "bool" too, since we just want on/off) - R. From rdreier at cisco.com Tue Jul 15 11:50:37 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 11:50:37 -0700 Subject: [ofa-general] Re: [PATCH] CMA: Notify ULP when CM exits the TIMEWAIT state. In-Reply-To: <000201c8e6a9$20f8fff0$27fc070a@amr.corp.intel.com> (Sean Hefty's message of "Tue, 15 Jul 2008 11:32:25 -0700") References: <12161313341500-git-send-email-amirv@mellanox.co.il> <000201c8e6a9$20f8fff0$27fc070a@amr.corp.intel.com> Message-ID: thanks, I applied this... >+ event.event = RDMA_CM_EVENT_TIMWAIT_EXIT; I converted "TIMWAIT" to "TIMEWAIT" in the event name though -- seems cleaner to me. From rdreier at cisco.com Tue Jul 15 12:11:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 12:11:21 -0700 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: <487B820E.4020002@mellanox.co.il> (Tziporet Koren's message of "Mon, 14 Jul 2008 19:42:54 +0300") References: <487B820E.4020002@mellanox.co.il> Message-ID: > > - I've been working on memory management extensions support for mlx4, > > but I'm not sure if it will be ready in time. Firmware for this > > may not be released for a while so it ain't urgent anyway. > We are testing the patches and we already have FW that enable them. > I agree its not urgent but it would be good to have it, so ULPs that > are interested can be tested over IB too. By the way, for outside of Mellanox, here is my patch that adds mem mgt extensions and local lkey support to mlx4. Comments appreciated... diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 299f208..0b191a4 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -637,6 +637,7 @@ repoll: case MLX4_OPCODE_SEND_IMM: wc->wc_flags |= IB_WC_WITH_IMM; case MLX4_OPCODE_SEND: + case MLX4_OPCODE_SEND_INVAL: wc->opcode = IB_WC_SEND; break; case MLX4_OPCODE_RDMA_READ: @@ -657,6 +658,12 @@ repoll: case MLX4_OPCODE_LSO: wc->opcode = IB_WC_LSO; break; + case MLX4_OPCODE_FMR: + wc->opcode = IB_WC_FAST_REG_MR; + break; + case MLX4_OPCODE_LOCAL_INVAL: + wc->opcode = IB_WC_LOCAL_INV; + break; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); @@ -667,6 +674,11 @@ repoll: wc->wc_flags = IB_WC_WITH_IMM; wc->ex.imm_data = cqe->immed_rss_invalid; break; + case MLX4_RECV_OPCODE_SEND_INVAL: + wc->opcode = IB_WC_RECV; + wc->wc_flags = IB_WC_WITH_INVALIDATE; + wc->ex.invalidate_rkey = be32_to_cpu(cqe->immed_rss_invalid); + break; case MLX4_RECV_OPCODE_SEND: wc->opcode = IB_WC_RECV; wc->wc_flags = 0; diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index bcf5064..38d6907 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -104,6 +104,12 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; if (dev->dev->caps.max_gso_sz) props->device_cap_flags |= IB_DEVICE_UD_TSO; + if (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_RESERVED_LKEY) + props->device_cap_flags |= IB_DEVICE_LOCAL_DMA_LKEY; + if ((dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_LOCAL_INV) && + (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_REMOTE_INV) && + (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_FAST_REG_WR)) + props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; @@ -127,6 +133,7 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->max_srq = dev->dev->caps.num_srqs - dev->dev->caps.reserved_srqs; props->max_srq_wr = dev->dev->caps.max_srq_wqes - 1; props->max_srq_sge = dev->dev->caps.max_srq_sge; + props->max_fast_reg_page_list_len = PAGE_SIZE / sizeof (u64); props->local_ca_ack_delay = dev->dev->caps.local_ca_ack_delay; props->atomic_cap = dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_ATOMIC ? IB_ATOMIC_HCA : IB_ATOMIC_NONE; @@ -565,6 +572,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) strlcpy(ibdev->ib_dev.name, "mlx4_%d", IB_DEVICE_NAME_MAX); ibdev->ib_dev.owner = THIS_MODULE; ibdev->ib_dev.node_type = RDMA_NODE_IB_CA; + ibdev->ib_dev.local_dma_lkey = dev->caps.reserved_lkey; ibdev->ib_dev.phys_port_cnt = dev->caps.num_ports; ibdev->ib_dev.num_comp_vectors = 1; ibdev->ib_dev.dma_device = &dev->pdev->dev; @@ -627,6 +635,9 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.get_dma_mr = mlx4_ib_get_dma_mr; ibdev->ib_dev.reg_user_mr = mlx4_ib_reg_user_mr; ibdev->ib_dev.dereg_mr = mlx4_ib_dereg_mr; + ibdev->ib_dev.alloc_fast_reg_mr = mlx4_ib_alloc_fast_reg_mr; + ibdev->ib_dev.alloc_fast_reg_page_list = mlx4_ib_alloc_fast_reg_page_list; + ibdev->ib_dev.free_fast_reg_page_list = mlx4_ib_free_fast_reg_page_list; ibdev->ib_dev.attach_mcast = mlx4_ib_mcg_attach; ibdev->ib_dev.detach_mcast = mlx4_ib_mcg_detach; ibdev->ib_dev.process_mad = mlx4_ib_process_mad; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index c4cf5b6..d26a913 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -83,6 +83,11 @@ struct mlx4_ib_mr { struct ib_umem *umem; }; +struct mlx4_ib_fast_reg_page_list { + struct ib_fast_reg_page_list ibfrpl; + dma_addr_t map; +}; + struct mlx4_ib_fmr { struct ib_fmr ibfmr; struct mlx4_fmr mfmr; @@ -199,6 +204,11 @@ static inline struct mlx4_ib_mr *to_mmr(struct ib_mr *ibmr) return container_of(ibmr, struct mlx4_ib_mr, ibmr); } +static inline struct mlx4_ib_fast_reg_page_list *to_mfrpl(struct ib_fast_reg_page_list *ibfrpl) +{ + return container_of(ibfrpl, struct mlx4_ib_fast_reg_page_list, ibfrpl); +} + static inline struct mlx4_ib_fmr *to_mfmr(struct ib_fmr *ibfmr) { return container_of(ibfmr, struct mlx4_ib_fmr, ibfmr); @@ -239,6 +249,11 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, u64 virt_addr, int access_flags, struct ib_udata *udata); int mlx4_ib_dereg_mr(struct ib_mr *mr); +struct ib_mr *mlx4_ib_alloc_fast_reg_mr(struct ib_pd *pd, + int max_page_list_len); +struct ib_fast_reg_page_list *mlx4_ib_alloc_fast_reg_page_list(struct ib_device *ibdev, + int page_list_len); +void mlx4_ib_free_fast_reg_page_list(struct ib_fast_reg_page_list *page_list); int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata); diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c index 68e9248..db2086f 100644 --- a/drivers/infiniband/hw/mlx4/mr.c +++ b/drivers/infiniband/hw/mlx4/mr.c @@ -183,6 +183,76 @@ int mlx4_ib_dereg_mr(struct ib_mr *ibmr) return 0; } +struct ib_mr *mlx4_ib_alloc_fast_reg_mr(struct ib_pd *pd, + int max_page_list_len) +{ + struct mlx4_ib_dev *dev = to_mdev(pd->device); + struct mlx4_ib_mr *mr; + int err; + + mr = kmalloc(sizeof *mr, GFP_KERNEL); + if (!mr) + return ERR_PTR(-ENOMEM); + + err = mlx4_mr_alloc(dev->dev, to_mpd(pd)->pdn, 0, 0, 0, + max_page_list_len, 0, &mr->mmr); + if (err) + goto err_free; + + err = mlx4_mr_enable(dev->dev, &mr->mmr); + if (err) + goto err_mr; + + return &mr->ibmr; + +err_mr: + mlx4_mr_free(dev->dev, &mr->mmr); + +err_free: + kfree(mr); + return ERR_PTR(err); +} + +struct ib_fast_reg_page_list *mlx4_ib_alloc_fast_reg_page_list(struct ib_device *ibdev, + int page_list_len) +{ + struct mlx4_ib_dev *dev = to_mdev(ibdev); + struct mlx4_ib_fast_reg_page_list *mfrpl; + int size = page_list_len * sizeof (u64); + + if (size > PAGE_SIZE) + return ERR_PTR(-EINVAL); + + mfrpl = kmalloc(sizeof *mfrpl, GFP_KERNEL); + if (!mfrpl) + return ERR_PTR(-ENOMEM); + + mfrpl->ibfrpl.page_list = dma_alloc_coherent(&dev->dev->pdev->dev, + size, &mfrpl->map, + GFP_KERNEL); + if (!mfrpl->ibfrpl.page_list) + goto err_free; + + WARN_ON(mfrpl->map & 0x3f); + + return &mfrpl->ibfrpl; + +err_free: + kfree(mfrpl); + return ERR_PTR(-ENOMEM); +} + +void mlx4_ib_free_fast_reg_page_list(struct ib_fast_reg_page_list *page_list) +{ + struct mlx4_ib_dev *dev = to_mdev(page_list->device); + struct mlx4_ib_fast_reg_page_list *mfrpl = to_mfrpl(page_list); + int size = page_list->max_page_list_len * sizeof (u64); + + dma_free_coherent(&dev->dev->pdev->dev, size, page_list->page_list, + mfrpl->map); + kfree(mfrpl); +} + struct ib_fmr *mlx4_ib_fmr_alloc(struct ib_pd *pd, int acc, struct ib_fmr_attr *fmr_attr) { diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 91590e7..47ec68d 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -78,6 +78,9 @@ static const __be32 mlx4_ib_opcode[] = { [IB_WR_RDMA_READ] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_READ), [IB_WR_ATOMIC_CMP_AND_SWP] = __constant_cpu_to_be32(MLX4_OPCODE_ATOMIC_CS), [IB_WR_ATOMIC_FETCH_AND_ADD] = __constant_cpu_to_be32(MLX4_OPCODE_ATOMIC_FA), + [IB_WR_SEND_WITH_INV] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_INVAL), + [IB_WR_LOCAL_INV] = __constant_cpu_to_be32(MLX4_OPCODE_LOCAL_INVAL), + [IB_WR_FAST_REG_MR] = __constant_cpu_to_be32(MLX4_OPCODE_FMR), }; static struct mlx4_ib_sqp *to_msqp(struct mlx4_ib_qp *mqp) @@ -987,6 +990,10 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pd = cpu_to_be32(to_mpd(ibqp->pd)->pdn); context->params1 = cpu_to_be32(MLX4_IB_ACK_REQ_FREQ << 28); + /* Set "fast registration enabled" for all kernel QPs */ + if (!qp->ibqp.uobject) + context->params1 |= cpu_to_be32(1 << 11); + if (attr_mask & IB_QP_RNR_RETRY) { context->params1 |= cpu_to_be32(attr->rnr_retry << 13); optpar |= MLX4_QP_OPTPAR_RNR_RETRY; @@ -1333,6 +1340,38 @@ static int mlx4_wq_overflow(struct mlx4_ib_wq *wq, int nreq, struct ib_cq *ib_cq return cur + nreq >= wq->max_post; } +static __be32 convert_access(int acc) +{ + return (acc & IB_ACCESS_REMOTE_ATOMIC ? cpu_to_be32(MLX4_WQE_FMR_PERM_ATOMIC) : 0) | + (acc & IB_ACCESS_REMOTE_WRITE ? cpu_to_be32(MLX4_WQE_FMR_PERM_REMOTE_WRITE) : 0) | + (acc & IB_ACCESS_REMOTE_READ ? cpu_to_be32(MLX4_WQE_FMR_PERM_REMOTE_READ) : 0) | + (acc & IB_ACCESS_LOCAL_WRITE ? cpu_to_be32(MLX4_WQE_FMR_PERM_LOCAL_WRITE) : 0) | + cpu_to_be32(MLX4_WQE_FMR_PERM_LOCAL_READ); +} + +static void set_fmr_seg(struct mlx4_wqe_fmr_seg *fseg, struct ib_send_wr *wr) +{ + struct mlx4_ib_fast_reg_page_list *mfrpl = to_mfrpl(wr->wr.fast_reg.page_list); + + fseg->flags = convert_access(wr->wr.fast_reg.access_flags); + fseg->mem_key = cpu_to_be32(wr->wr.fast_reg.rkey); + fseg->buf_list = cpu_to_be64(mfrpl->map); + fseg->start_addr = cpu_to_be64(wr->wr.fast_reg.iova_start); + fseg->reg_len = cpu_to_be64(wr->wr.fast_reg.length); + fseg->offset = 0; /* XXX -- is this just for ZBVA? */ + fseg->page_size = cpu_to_be32(wr->wr.fast_reg.page_shift); + fseg->reserved[0] = 0; + fseg->reserved[1] = 0; +} + +static void set_local_inv_seg(struct mlx4_wqe_local_inval_seg *iseg, u32 rkey) +{ + iseg->flags = 0; + iseg->mem_key = cpu_to_be32(rkey); + iseg->guest_id = 0; + iseg->pa = 0; +} + static __always_inline void set_raddr_seg(struct mlx4_wqe_raddr_seg *rseg, u64 remote_addr, u32 rkey) { @@ -1434,6 +1473,21 @@ static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, return 0; } +static __be32 send_ieth(struct ib_send_wr *wr) +{ + switch (wr->opcode) { + case IB_WR_SEND_WITH_IMM: + case IB_WR_RDMA_WRITE_WITH_IMM: + return wr->ex.imm_data; + + case IB_WR_SEND_WITH_INV: + return cpu_to_be32(wr->ex.invalidate_rkey); + + default: + return 0; + } +} + int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, struct ib_send_wr **bad_wr) { @@ -1480,11 +1534,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, MLX4_WQE_CTRL_TCP_UDP_CSUM) : 0) | qp->sq_signal_bits; - if (wr->opcode == IB_WR_SEND_WITH_IMM || - wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) - ctrl->imm = wr->ex.imm_data; - else - ctrl->imm = 0; + ctrl->imm = send_ieth(wr); wqe += sizeof *ctrl; size = sizeof *ctrl / 16; @@ -1516,6 +1566,18 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, size += sizeof (struct mlx4_wqe_raddr_seg) / 16; break; + case IB_WR_LOCAL_INV: + set_local_inv_seg(wqe, wr->ex.invalidate_rkey); + wqe += sizeof (struct mlx4_wqe_local_inval_seg); + size += sizeof (struct mlx4_wqe_local_inval_seg) / 16; + break; + + case IB_WR_FAST_REG_MR: + set_fmr_seg(wqe, wr); + wqe += sizeof (struct mlx4_wqe_fmr_seg); + size += sizeof (struct mlx4_wqe_fmr_seg) / 16; + break; + default: /* No extra segments required for sends */ break; diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index 2b5006b..1180fca 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -198,7 +198,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_DEV_CAP_C_MPT_ENTRY_SZ_OFFSET 0x8e #define QUERY_DEV_CAP_MTT_ENTRY_SZ_OFFSET 0x90 #define QUERY_DEV_CAP_D_MPT_ENTRY_SZ_OFFSET 0x92 -#define QUERY_DEV_CAP_BMME_FLAGS_OFFSET 0x97 +#define QUERY_DEV_CAP_BMME_FLAGS_OFFSET 0x94 #define QUERY_DEV_CAP_RSVD_LKEY_OFFSET 0x98 #define QUERY_DEV_CAP_MAX_ICM_SZ_OFFSET 0xa0 @@ -373,12 +373,8 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) } } - if (dev_cap->bmme_flags & 1) - mlx4_dbg(dev, "Base MM extensions: yes " - "(flags %d, rsvd L_Key %08x)\n", - dev_cap->bmme_flags, dev_cap->reserved_lkey); - else - mlx4_dbg(dev, "Base MM extensions: no\n"); + mlx4_dbg(dev, "Base MM extensions: flags %08x, rsvd L_Key %08x\n", + dev_cap->bmme_flags, dev_cap->reserved_lkey); /* * Each UAR has 4 EQ doorbells; so if a UAR is reserved, then diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index a0e046c..fbf0e22 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -98,7 +98,7 @@ struct mlx4_dev_cap { int cmpt_entry_sz; int mtt_entry_sz; int resize_srq; - u8 bmme_flags; + u32 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; int max_gso_sz; diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index d373601..8e1d24c 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -158,6 +158,8 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.max_msg_sz = dev_cap->max_msg_sz; dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; + dev->caps.bmme_flags = dev_cap->bmme_flags; + dev->caps.reserved_lkey = dev_cap->reserved_lkey; dev->caps.stat_rate_support = dev_cap->stat_rate_support; dev->caps.max_gso_sz = dev_cap->max_gso_sz; diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 03a9abc..66c6f84 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -47,7 +47,7 @@ struct mlx4_mpt_entry { __be32 flags; __be32 qpn; __be32 key; - __be32 pd; + __be32 pd_flags; __be64 start; __be64 length; __be32 lkey; @@ -61,11 +61,15 @@ struct mlx4_mpt_entry { } __attribute__((packed)); #define MLX4_MPT_FLAG_SW_OWNS (0xfUL << 28) +#define MLX4_MPT_FLAG_FREE (0x3UL << 28) #define MLX4_MPT_FLAG_MIO (1 << 17) #define MLX4_MPT_FLAG_BIND_ENABLE (1 << 15) #define MLX4_MPT_FLAG_PHYSICAL (1 << 9) #define MLX4_MPT_FLAG_REGION (1 << 8) +#define MLX4_MPT_PD_FLAG_FAST_REG (1 << 26) +#define MLX4_MPT_PD_FLAG_EN_INV (3 << 24) + #define MLX4_MTT_FLAG_PRESENT 1 #define MLX4_MPT_STATUS_SW 0xF0 @@ -314,21 +318,30 @@ int mlx4_mr_enable(struct mlx4_dev *dev, struct mlx4_mr *mr) memset(mpt_entry, 0, sizeof *mpt_entry); - mpt_entry->flags = cpu_to_be32(MLX4_MPT_FLAG_SW_OWNS | - MLX4_MPT_FLAG_MIO | + mpt_entry->flags = cpu_to_be32(MLX4_MPT_FLAG_MIO | MLX4_MPT_FLAG_REGION | mr->access); mpt_entry->key = cpu_to_be32(key_to_hw_index(mr->key)); - mpt_entry->pd = cpu_to_be32(mr->pd); + mpt_entry->pd_flags = cpu_to_be32(mr->pd | MLX4_MPT_PD_FLAG_EN_INV); mpt_entry->start = cpu_to_be64(mr->iova); mpt_entry->length = cpu_to_be64(mr->size); mpt_entry->entity_size = cpu_to_be32(mr->mtt.page_shift); + if (mr->mtt.order < 0) { mpt_entry->flags |= cpu_to_be32(MLX4_MPT_FLAG_PHYSICAL); mpt_entry->mtt_seg = 0; - } else + } else { mpt_entry->mtt_seg = cpu_to_be64(mlx4_mtt_addr(dev, &mr->mtt)); + } + + if (mr->mtt.order >= 0 && mr->mtt.page_shift == 0) { + /* fast register MR in free state */ + mpt_entry->flags |= cpu_to_be32(MLX4_MPT_FLAG_FREE); + mpt_entry->pd_flags |= cpu_to_be32(MLX4_MPT_PD_FLAG_FAST_REG); + } else { + mpt_entry->flags |= cpu_to_be32(MLX4_MPT_FLAG_SW_OWNS); + } err = mlx4_SW2HW_MPT(dev, mailbox, key_to_hw_index(mr->key) & (dev->caps.num_mpts - 1)); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 81b3dd5..655ea0d 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -68,6 +68,14 @@ enum { MLX4_DEV_CAP_FLAG_UD_MCAST = 1 << 21 }; +enum { + MLX4_BMME_FLAG_LOCAL_INV = 1 << 6, + MLX4_BMME_FLAG_REMOTE_INV = 1 << 7, + MLX4_BMME_FLAG_TYPE_2_WIN = 1 << 9, + MLX4_BMME_FLAG_RESERVED_LKEY = 1 << 10, + MLX4_BMME_FLAG_FAST_REG_WR = 1 << 11, +}; + enum mlx4_event { MLX4_EVENT_TYPE_COMP = 0x00, MLX4_EVENT_TYPE_PATH_MIG = 0x01, @@ -184,6 +192,8 @@ struct mlx4_caps { u32 max_msg_sz; u32 page_size_cap; u32 flags; + u32 bmme_flags; + u32 reserved_lkey; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; int max_gso_sz; diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 7f128b2..5cb34fb 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -233,6 +233,14 @@ struct mlx4_wqe_bind_seg { __be64 length; }; +enum { + MLX4_WQE_FMR_PERM_LOCAL_READ = 1 << 27, + MLX4_WQE_FMR_PERM_LOCAL_WRITE = 1 << 28, + MLX4_WQE_FMR_PERM_REMOTE_READ = 1 << 29, + MLX4_WQE_FMR_PERM_REMOTE_WRITE = 1 << 30, + MLX4_WQE_FMR_PERM_ATOMIC = 1 << 31 +}; + struct mlx4_wqe_fmr_seg { __be32 flags; __be32 mem_key; @@ -255,11 +263,9 @@ struct mlx4_wqe_fmr_ext_seg { }; struct mlx4_wqe_local_inval_seg { - u8 flags; - u8 reserved1[3]; + __be32 flags; __be32 mem_key; - u8 reserved2[3]; - u8 guest_id; + __be32 guest_id; __be64 pa; }; From jsquyres at cisco.com Tue Jul 15 12:14:37 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Tue, 15 Jul 2008 15:14:37 -0400 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <000101c8e69c$b35025c0$27fc070a@amr.corp.intel.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> <487B8727.20003@opengridcomputing.com> <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> <487C816C.7040301@mellanox.co.il> <000101c8e69c$b35025c0$27fc070a@amr.corp.intel.com> Message-ID: On Jul 15, 2008, at 1:03 PM, Sean Hefty wrote: >>> - Any estimate on the "ibcm_listen() sometimes failing" issue? >>> (I >>> don't think there's a ticket filed against this yet) >>> >> I don't think anything was done here - Sean? > > I have been working with Jeff and the ompi developers directly on a > different > thread, but I don't think that ofa general is copied on it. There's a > possibility that ompi is colliding with the SDP listen space, and > we're > exploring that option. Yep-- sorry for the cross posting; this one was posted before we started the thread yesterday on the OMPI list about the same issue. There's a high probability that we're doing something wrong in OMPI's IBCM support. -- Jeff Squyres Cisco Systems From or.gerlitz at gmail.com Tue Jul 15 12:20:56 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 15 Jul 2008 22:20:56 +0300 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: References: <487B820E.4020002@mellanox.co.il> Message-ID: <15ddcffd0807151220m37204055rf6890037b445bd34@mail.gmail.com> On Tue, Jul 15, 2008 at 10:11 PM, Roland Dreier wrote: > >> - I've been working on memory management extensions support for mlx4, > >> but I'm not sure if it will be ready in time. Firmware for this > >> may not be released for a while so it ain't urgent anyway. > > > We are testing the patches and we already have FW that enable them. > > I agree its not urgent but it would be good to have it, so ULPs that > > are interested can be tested over IB too. > > By the way, for outside of Mellanox, here is my patch that adds mem mgt > extensions and local lkey support to mlx4. Comments appreciated... > So just to make sure, firmware 2.5 does not support IB BMME and if I want to test with this patch I need to get a special firmware drop from Mellanox? Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Jul 15 12:19:41 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 12:19:41 -0700 Subject: [ofa-general] [PATCH] IB/mlx4: Rename struct mlx4_lso_seg to mlx4_wqe_lso_seg Message-ID: Make the struct name consistent with other WQE segment struct types defined in . Signed-off-by: Roland Dreier --- diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 89eb6cb..bda0859 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -1395,7 +1395,7 @@ static void __set_data_seg(struct mlx4_wqe_data_seg *dseg, struct ib_sge *sg) dseg->addr = cpu_to_be64(sg->addr); } -static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, +static int build_lso_seg(struct mlx4_wqe_lso_seg *wqe, struct ib_send_wr *wr, struct mlx4_ib_qp *qp, unsigned *lso_seg_len) { unsigned halign = ALIGN(sizeof *wqe + wr->wr.ud.hlen, 16); diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 7f128b2..f02e9ed 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -219,7 +219,7 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; -struct mlx4_lso_seg { +struct mlx4_wqe_lso_seg { __be32 mss_hdr_size; __be32 header[0]; }; From rdreier at cisco.com Tue Jul 15 12:36:32 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 12:36:32 -0700 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: <15ddcffd0807151220m37204055rf6890037b445bd34@mail.gmail.com> (Or Gerlitz's message of "Tue, 15 Jul 2008 22:20:56 +0300") References: <487B820E.4020002@mellanox.co.il> <15ddcffd0807151220m37204055rf6890037b445bd34@mail.gmail.com> Message-ID: > So just to make sure, firmware 2.5 does not support IB BMME and if I want to > test with this patch I need to get a special firmware drop from Mellanox? Right: in the test + if ((dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_LOCAL_INV) && + (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_REMOTE_INV) && + (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_FAST_REG_WR)) + props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; you will see that not all the bmme_flags required are set by FW 2.5. From or.gerlitz at gmail.com Tue Jul 15 12:16:38 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 15 Jul 2008 22:16:38 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) In-Reply-To: References: <200807151745.53152.jackm@dev.mellanox.co.il> Message-ID: <15ddcffd0807151216h3fe364bcx5d164ffa15694117@mail.gmail.com> On Tue, Jul 15, 2008 at 9:41 PM, Roland Dreier wrote: > thanks, applied... a few comments: > Jack, Roland, I'd be happy to get educated first, why this patch/param is needed at all? and second, why ofed 1.3 is shipped with this patch/param but an opposite default value...? As for the first point, I understand that telling the HCA "don't spend time on arbitration between VLs" may reduce some ns from the latency, but I don't think it worth the hassle of managing this module param, with connectx and PCIe gen2 I recently saw 0.99 us latency (rtt/2) so I guess you can't go much below that with this patch, or is there another reason for its existence? Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From itukagno at THENAMARIS.GR Tue Jul 15 14:19:52 2008 From: itukagno at THENAMARIS.GR (Kubin) Date: Tue, 15 Jul 2008 16:19:52 -0500 Subject: [ofa-general] Plastic surgery gone wrong Message-ID: Military insurgence in Afghan, US troops killed http://thegladiator.altervista.org/about.html -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ctung at NetEffect.com Tue Jul 15 13:31:37 2008 From: ctung at NetEffect.com (Chien Tung) Date: Tue, 15 Jul 2008 15:31:37 -0500 Subject: [ofa-general] Question In-Reply-To: <99863D2ED484D449811D97A4C44C9CBD82FAF0@EPEXCH2.qlogic.org> References: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org><487CC4B8.5080506@mellanox.co.il> <99863D2ED484D449811D97A4C44C9CBD82FAF0@EPEXCH2.qlogic.org> Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC083118DD@venom2> > We have found that causes openibd to be started before > networking and therefore the NodeDescription, when returned > from the SM, does not always contain the hostname of the > system when ibhosts is run. > > Is this a problem for anyone else? We have a reported issue on RHEL4U4 with device file not getting created properly. We believe it is related to this change but have not fully root cause the problem. Chien From rdreier at cisco.com Tue Jul 15 13:56:50 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 13:56:50 -0700 Subject: [ofa-general] [PATCH] RDMA/ucma: BKL is not needed for ucma_open() Message-ID: Remove explicit lock_kernel() calls and document why the code is safe. Signed-off-by: Roland Dreier --- I'll merge this through my tree unless someone sees something wrong with it... drivers/infiniband/core/ucma.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c index 195f973..b41dd26 100644 --- a/drivers/infiniband/core/ucma.c +++ b/drivers/infiniband/core/ucma.c @@ -38,7 +38,6 @@ #include #include #include -#include #include #include @@ -1149,6 +1148,14 @@ static unsigned int ucma_poll(struct file *filp, struct poll_table_struct *wait) return mask; } +/* + * ucma_open() does not need the BKL: + * + * - no global state is referred to; + * - there is no ioctl method to race against; + * - no further module initialization is required for open to work + * after the device is registered. + */ static int ucma_open(struct inode *inode, struct file *filp) { struct ucma_file *file; @@ -1157,7 +1164,6 @@ static int ucma_open(struct inode *inode, struct file *filp) if (!file) return -ENOMEM; - lock_kernel(); INIT_LIST_HEAD(&file->event_list); INIT_LIST_HEAD(&file->ctx_list); init_waitqueue_head(&file->poll_wait); @@ -1165,7 +1171,6 @@ static int ucma_open(struct inode *inode, struct file *filp) filp->private_data = file; file->filp = filp; - unlock_kernel(); return 0; } -- 1.5.6.2 From rdreier at cisco.com Tue Jul 15 13:57:25 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 13:57:25 -0700 Subject: [ofa-general] [PATCH] RDMA/ucm: BKL is not needed for ib_ucm_open() Message-ID: Remove explicit cycle_kernel_lock() call and document why the code is safe. Signed-off-by: Roland Dreier --- And merge this too... drivers/infiniband/core/ucm.c | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c index 9494005..e603736 100644 --- a/drivers/infiniband/core/ucm.c +++ b/drivers/infiniband/core/ucm.c @@ -43,7 +43,6 @@ #include #include #include -#include #include @@ -1154,11 +1153,18 @@ static unsigned int ib_ucm_poll(struct file *filp, return mask; } +/* + * ib_ucm_open() does not need the BKL: + * + * - no global state is referred to; + * - there is no ioctl method to race against; + * - no further module initialization is required for open to work + * after the device is registered. + */ static int ib_ucm_open(struct inode *inode, struct file *filp) { struct ib_ucm_file *file; - cycle_kernel_lock(); file = kmalloc(sizeof(*file), GFP_KERNEL); if (!file) return -ENOMEM; -- 1.5.6.2 From 23bad9 at flash.net Tue Jul 15 14:24:05 2008 From: 23bad9 at flash.net (kayne ho-ming) Date: Tue, 15 Jul 2008 21:24:05 +0000 Subject: [ofa-general] to general Message-ID: <000401c8e6d0$0594fd6d$4405dc9e@etwvo> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.rahwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From rdreier at cisco.com Tue Jul 15 19:39:43 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 15 Jul 2008 19:39:43 -0700 Subject: [ofa-general][PATCH 2/11 v3] mlx4: Pre reserved Qp regions. In-Reply-To: <4874BE85.6070108@mellanox.co.il> (Yevgeny Petrilin's message of "Wed, 09 Jul 2008 16:35:01 +0300") References: <4874BE85.6070108@mellanox.co.il> Message-ID: > +int mlx4_qp_get_region(struct mlx4_dev *dev, > + enum qp_region region, > + int *base_qpn, int *cnt) I don't seem to see any call to this in the mlx4_en driver you posted to netdev, nor to MLX4_QP_REGION_ETH_ADDR. Can you explain what this mechanism is used for? Is this only for the FC driver? From keshetti.mahesh at gmail.com Tue Jul 15 22:38:09 2008 From: keshetti.mahesh at gmail.com (Keshetti Mahesh) Date: Wed, 16 Jul 2008 11:08:09 +0530 Subject: [ofa-general] IB bonding documentation Message-ID: <829ded920807152238rd6f0e03q8156e3d95849d5fe@mail.gmail.com> 'ib-bonding.txt' file in the "docs" folder of OFED-1.3 release mentions ib-bonding as a HA solution for IPoIB interfaces. But it does not mention anything about link aggregation. My question is whether ethernet channel bonding is completely adopted to work with IPoIB interfaces or only parts of it are adopted ? If all the 'mode's (balance-rr, balance-xor etc.) are supported in ib-bonding then how they are supported (I've read in ethernet channel bonding documentation that balance-rr, balance-xor etc. modes operation requires 'switch' support) ? -Mahesh From anuj01 at gmail.com Tue Jul 15 22:38:26 2008 From: anuj01 at gmail.com (=?UTF-8?B?4KSF4KSo4KWB4KSc?=) Date: Wed, 16 Jul 2008 11:08:26 +0530 Subject: [ofa-general] hi - configuring libmthca Message-ID: Hi When m running script ./configure in libmthca, then an error is generated : . . checking for ibv_get_device_list in -libverbs... yes checking infiniband/driver.h usability... yes checking infiniband/driver.h presence... no configure: WARNING: infiniband/driver.h: accepted by the compiler, rejected by the preprocessor! configure: WARNING: infiniband/driver.h: proceeding with the compiler's result checking for infiniband/driver.h... yes checking for ANSI C header files... (cached) yes checking sysfs/libsysfs.h usability... yes checking sysfs/libsysfs.h presence... yes checking for sysfs/libsysfs.h... yes checking valgrind/memcheck.h usability... yes checking valgrind/memcheck.h presence... yes checking for valgrind/memcheck.h... yes checking for an ANSI C-conforming const... yes checking for long... yes checking size of long... configure: error: cannot compute sizeof (long), 77 How can this be resolved? Regards, Anuj Aggarwal From eli at dev.mellanox.co.il Tue Jul 15 23:57:46 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 16 Jul 2008 09:57:46 +0300 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support In-Reply-To: References: <20080715161251.GA16299@mtls03> Message-ID: <20080716065746.GA720@mtls03> On Tue, Jul 15, 2008 at 11:11:25AM -0700, Shirley Ma wrote: > That's a good idea. We talked about this internally. It benefits both UD > and CM mode. The patch series is designed to support only UD mode. Moreover, mlx4 hardware supports this only for UD mode. > > Does this RCA share same CQ? Or different RCA could use different CQ/EQ > per core? > RCA stands for a a number of QPs, each connected to its private CQ, and each such CQ reports to different EQ. The differnet EQs generate different vector interrupts. From olga.shern at gmail.com Wed Jul 16 01:55:04 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Wed, 16 Jul 2008 11:55:04 +0300 Subject: [ofa-general] IB bonding documentation In-Reply-To: <829ded920807152238rd6f0e03q8156e3d95849d5fe@mail.gmail.com> References: <829ded920807152238rd6f0e03q8156e3d95849d5fe@mail.gmail.com> Message-ID: Hi Mahesh, ib-bonding for IPoIB is working only in ACTIVE - PASSIVE mode. No link aggregation mode is supported. Olga On 7/16/08, Keshetti Mahesh wrote: > 'ib-bonding.txt' file in the "docs" folder of OFED-1.3 release mentions > ib-bonding as a HA solution for IPoIB interfaces. But it does not mention > anything about link aggregation. My question is whether ethernet channel > bonding is completely adopted to work with IPoIB interfaces or only parts > of it are adopted ? > > If all the 'mode's (balance-rr, balance-xor etc.) are supported in > ib-bonding then > how they are supported (I've read in ethernet channel bonding > documentation that > balance-rr, balance-xor etc. modes operation requires 'switch' support) ? > > -Mahesh > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From keshetti.mahesh at gmail.com Wed Jul 16 02:07:31 2008 From: keshetti.mahesh at gmail.com (Keshetti Mahesh) Date: Wed, 16 Jul 2008 14:37:31 +0530 Subject: [ofa-general] IB bonding documentation In-Reply-To: References: <829ded920807152238rd6f0e03q8156e3d95849d5fe@mail.gmail.com> Message-ID: <829ded920807160207y5da73498w5a4ef1896adb1881@mail.gmail.com> > ib-bonding for IPoIB is working only in ACTIVE - PASSIVE mode. > No link aggregation mode is supported. Thanks for the reply. I remember there was a IPOIB-HA tool included in OFED before which does the same. What happened to that ? Is there any work going on to support all 'mode's of ib-bonding for IPoIB? Finally, Is it required to connect all slaves to same switch to achieve HA with ib-bonding ? -Mahesh From olga.shern at gmail.com Wed Jul 16 02:18:33 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Wed, 16 Jul 2008 12:18:33 +0300 Subject: [ofa-general] IB bonding documentation In-Reply-To: <829ded920807160207y5da73498w5a4ef1896adb1881@mail.gmail.com> References: <829ded920807152238rd6f0e03q8156e3d95849d5fe@mail.gmail.com> <829ded920807160207y5da73498w5a4ef1896adb1881@mail.gmail.com> Message-ID: On 7/16/08, Keshetti Mahesh wrote: > > ib-bonding for IPoIB is working only in ACTIVE - PASSIVE mode. > > No link aggregation mode is supported. > > > Thanks for the reply. I remember there was a IPOIB-HA tool > included in OFED before which does the same. What happened to that ? > Bonding is the best way to achieve HA with IPoIB. IPOIB-HA tool no longer supported. > Is there any work going on to support all 'mode's of ib-bonding for IPoIB? > Yes, we are working on this. Not all modes but to enable Load balancing. > Finally, Is it required to connect all slaves to same switch to achieve HA with > ib-bonding ? > Yes, currently HA supported only with one Infiniband subnet. We also working on this, to support HA on two different Infiniband subnets. > -Mahesh > Olga From 4flowers at feely-associates.com Wed Jul 16 00:37:55 2008 From: 4flowers at feely-associates.com (broderick manny) Date: Wed, 16 Jul 2008 07:37:55 +0000 Subject: [ofa-general] to general Message-ID: <000501c8e725$035c0e42$8dca7392@kemgkc> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.rajwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From vlad at lists.openfabrics.org Wed Jul 16 02:43:10 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Wed, 16 Jul 2008 02:43:10 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080716-0200 daily build status Message-ID: <20080716094310.B9139E60897@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ia64 with linux-2.6.26-rc6 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.24 Failed: Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-1.2798.fc6_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-1.2798.fc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-53.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-53.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-53.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -fasynchronous-unwind-tables -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_driver)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: error: unknown field 'groups' specified in initializer /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:155: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.20_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.20' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.18-93.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-93.el5_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2616: error: implicit declaration of function 'pid_nr' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_signal_procs': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:2671: error: implicit declaration of function 'kill_pid' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1260: warning: format '%Lx' expects type 'long long unsigned int', but argument 6 has type 'long unsigned int' /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c: In function 'ipath_reset_availshadow': /home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.c:1459: warning: format '%Lx' expects type 'long long unsigned int', but argument 4 has type 'u64' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath/ipath_driver.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080716-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Wed Jul 16 02:44:44 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 16 Jul 2008 12:44:44 +0300 Subject: [ofa-general] RE: [PATCH] CMA: Notify ULP when CM exits TIMEWAIT - interaction with the addr change event In-Reply-To: <000201c8e6a9$20f8fff0$27fc070a@amr.corp.intel.com> References: <12161313341500-git-send-email-amirv@mellanox.co.il> <000201c8e6a9$20f8fff0$27fc070a@amr.corp.intel.com> Message-ID: <487DC30C.6040102@voltaire.com> Sean Hefty wrote: > Roland, this obviously isn't critical, but it would be nice to get into 2.6.27 > if possible, to include it in the next release of the librdmacm. >> --- a/include/rdma/rdma_cm.h >> +++ b/include/rdma/rdma_cm.h >> @@ -57,7 +57,8 @@ enum rdma_cm_event_type { >> RDMA_CM_EVENT_DISCONNECTED, >> RDMA_CM_EVENT_DEVICE_REMOVAL, >> RDMA_CM_EVENT_MULTICAST_JOIN, >> - RDMA_CM_EVENT_MULTICAST_ERROR >> + RDMA_CM_EVENT_MULTICAST_ERROR, >> + RDMA_CM_EVENT_TIMWAIT_EXIT >> }; librdmacm has already the address change event being the next in the enum after the multicast error event, so merging this patch to the kernel before the address change one will make the rdma cm event enums different between user and kernel. Roland, would it be possible to apply this patch on top of the address change one? I have checked and Linus has not merged yet the bonding patch that allows the rdmacm addr change patch to get compiled, so its going to take few more days, I guess. Or. From ogerlitz at voltaire.com Wed Jul 16 04:12:32 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 16 Jul 2008 14:12:32 +0300 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support In-Reply-To: <20080716065746.GA720@mtls03> References: <20080715161251.GA16299@mtls03> <20080716065746.GA720@mtls03> Message-ID: <487DD7A0.4040709@voltaire.com> Eli Cohen wrote: > RCA stands for a a number of QPs, each connected to its private CQ, and each such CQ reports to different EQ. The differnet EQs generate different vector interrupts. > In tests I made on my machine I got the following results with netperf TCP, 4 streams. > Without RCA: 730 MB/s > With RCA: 970 MB/s So the performance improvement was seen with both the multiple EQs and RCA patches applied? I think that it would be better to know the improvement achieved by applying each one of them. Or. From webmaster at online-poker-inside.com Wed Jul 16 06:44:29 2008 From: webmaster at online-poker-inside.com (webmaster at online-poker-inside.com) Date: Wed, 16 Jul 2008 08:44:29 -0500 Subject: [ofa-general] Interesting offer from online-poker-inside.com to lists.openfabrics.org Message-ID: Dear Webmaster, My name is James and I represent online-poker-inside.com. I have a business proposal for you regarding lists.openfabrics.org. Please get back to me if you are interested and I will explain further so we can make some business together. Kind Regards, James. From olga.shern at gmail.com Wed Jul 16 07:11:26 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Wed, 16 Jul 2008 17:11:26 +0300 Subject: [ofa-general] IB bonding documentation In-Reply-To: References: <829ded920807152238rd6f0e03q8156e3d95849d5fe@mail.gmail.com> <829ded920807160207y5da73498w5a4ef1896adb1881@mail.gmail.com> Message-ID: On 7/16/08, Olga Shern (Voltaire) wrote: > On 7/16/08, Keshetti Mahesh wrote: > > > ib-bonding for IPoIB is working only in ACTIVE - PASSIVE mode. > > > No link aggregation mode is supported. > > > > > > Thanks for the reply. I remember there was a IPOIB-HA tool > > included in OFED before which does the same. What happened to that ? > > > Bonding is the best way to achieve HA with IPoIB. > IPOIB-HA tool no longer supported. > > > Is there any work going on to support all 'mode's of ib-bonding for IPoIB? > > > Yes, we are working on this. Not all modes but to enable Load balancing. > > > Finally, Is it required to connect all slaves to same switch to achieve HA with > > ib-bonding ? > > > Yes, currently HA supported only with one Infiniband subnet. > We also working on this, to support HA on two different Infiniband subnets. > You can connect HCA different ports (slaves) to different switches, but they should be in the same Infiniband fabric, that what I meant :) > > -Mahesh > > > > Olga > From eli at dev.mellanox.co.il Wed Jul 16 08:01:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 16 Jul 2008 18:01:25 +0300 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support In-Reply-To: <487DD7A0.4040709@voltaire.com> References: <20080715161251.GA16299@mtls03> <20080716065746.GA720@mtls03> <487DD7A0.4040709@voltaire.com> Message-ID: <20080716150125.GE7921@mtls03> On Wed, Jul 16, 2008 at 02:12:32PM +0300, Or Gerlitz wrote: > So the performance improvement was seen with both the multiple EQs and RCA > patches applied? Yes. > I think that it would be better to know the improvement > achieved by applying each one of them. > But the when I said "without RCA" it was with all the patches applied but RCA was disabled through the module param. So the gain in performance can be attibuted to RCA. From jackm at dev.mellanox.co.il Wed Jul 16 08:11:10 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 16 Jul 2008 18:11:10 +0300 Subject: [ofa-general] Re: [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) In-Reply-To: <15ddcffd0807151216h3fe364bcx5d164ffa15694117@mail.gmail.com> References: <200807151745.53152.jackm@dev.mellanox.co.il> <15ddcffd0807151216h3fe364bcx5d164ffa15694117@mail.gmail.com> Message-ID: <200807161811.11145.jackm@dev.mellanox.co.il> On Tuesday 15 July 2008 22:16, Or Gerlitz wrote: > I'd be happy to get educated first, why this patch/param is needed at all? Without this patch, QoS will not work at all. > and second, why ofed 1.3 is shipped with this patch/param but an opposite > default value...? > We noticed (after ofed 1.3 was released) that setting the QoS bit in the HCA resulted in using more HW resources. Since Qos is usually not used we decided to disable the feature by default, and allow enabling it when it is needed. In ofed 1.3.1 the default value was therefore changed to be OFF. - Jack From yevgenyp at mellanox.co.il Wed Jul 16 08:12:02 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Wed, 16 Jul 2008 18:12:02 +0300 Subject: [ofa-general][PATCH 2/11 v3] mlx4: Pre reserved Qp regions. In-Reply-To: References: <4874BE85.6070108@mellanox.co.il> Message-ID: <487E0FC2.600@mellanox.co.il> Roland Dreier wrote: > > +int mlx4_qp_get_region(struct mlx4_dev *dev, > > + enum qp_region region, > > + int *base_qpn, int *cnt) > > I don't seem to see any call to this in the mlx4_en driver you posted to > netdev, nor to MLX4_QP_REGION_ETH_ADDR. Can you explain what this > mechanism is used for? Is this only for the FC driver? > This mechanism is currently in use by the FC driver. It can be used by the mlx4_en driver when we add the address steering support to it. In address steering mode, each combination of Mac-Vlan is mapped to a QP. Thus we will need a relatively big consecutive range of QPs. Those ranges are reserved and a call for mlx4_qp_get_region with the required region type (eth_addr, fc_addr, fc_exch) will give the required range. Yevgeny From 3abc at xxx.co.jp Wed Jul 16 08:05:24 2008 From: 3abc at xxx.co.jp (claiborne fereydoo) Date: Wed, 16 Jul 2008 15:05:24 +0000 Subject: [ofa-general] to general Message-ID: <000901c8e764$0433e06c$a9b291bf@njsjeqoi> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.rajwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From john.russo at qlogic.com Wed Jul 16 11:36:02 2008 From: john.russo at qlogic.com (John Russo) Date: Wed, 16 Jul 2008 13:36:02 -0500 Subject: [ofa-general] Question regarding chkconfig In-Reply-To: <5E701717F2B2ED4EA60F87C8AA57B7CC083118DD@venom2> References: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org><487CC4B8.5080506@mellanox.co.il> <99863D2ED484D449811D97A4C44C9CBD82FAF0@EPEXCH2.qlogic.org> <5E701717F2B2ED4EA60F87C8AA57B7CC083118DD@venom2> Message-ID: <99863D2ED484D449811D97A4C44C9CBD82FB9D@EPEXCH2.qlogic.org> It looks like this issue is important to a few of us. Can anyone comment on these concerns? -----Original Message----- From: Chien Tung [mailto:ctung at NetEffect.com] Sent: Tuesday, July 15, 2008 4:32 PM To: John Russo; vlad at mellanox.co.il Cc: general at lists.openfabrics.org Subject: RE: [ofa-general] Question > We have found that causes openibd to be started before > networking and therefore the NodeDescription, when returned > from the SM, does not always contain the hostname of the > system when ibhosts is run. > > Is this a problem for anyone else? We have a reported issue on RHEL4U4 with device file not getting created properly. We believe it is related to this change but have not fully root cause the problem. Chien From rdreier at cisco.com Wed Jul 16 12:12:33 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 16 Jul 2008 12:12:33 -0700 Subject: [ofa-general] Re: [PATCH 11/12 V2] mlx4: XRC receive-only QPs In-Reply-To: <200807101852.06851.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Thu, 10 Jul 2008 18:52:06 +0300") References: <200807101852.06851.jackm@dev.mellanox.co.il> Message-ID: I have one very specific question: > --- a/drivers/infiniband/hw/mlx4/cq.c > +++ b/drivers/infiniband/hw/mlx4/cq.c > @@ -174,7 +174,7 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector > if (entries < 1 || entries > dev->dev->caps.max_cqes) > return ERR_PTR(-EINVAL); > > - cq = kmalloc(sizeof *cq, GFP_KERNEL); > + cq = kzalloc(sizeof *cq, GFP_KERNEL); > if (!cq) > return ERR_PTR(-ENOMEM); why is this change made in this patch? And a more general question: why do these kernel receive-only QPs affect device-dependent low-level driver code at all. This whole patch seems to be just doing bookkeeping that seems like it could be done in generic code, and it leads to layering problems like: + cq = mlx4_ib_create_cq(ibdev, 1, 0, NULL, NULL); + if (IS_ERR(cq)) { + err = PTR_ERR(cq); + goto err_cq; + } + cq->device = ibdev; + cq->comp_handler = mlx4_dummy_comp_handler; ie mlx4 ends up duplicating code from ib_create_cq() for no reason I can understand. I'm probably missing something but why can't all or at least most of this be done in the generic IB code? I don't see anything related to device HW interface anywhere in this patch. Create a new ib_xrc_kern_recv_qp structure or other helpers if need be I guess. - R. From weiny2 at llnl.gov Wed Jul 16 12:47:43 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 16 Jul 2008 12:47:43 -0700 Subject: [ofa-general] Question regarding chkconfig In-Reply-To: <99863D2ED484D449811D97A4C44C9CBD82FB9D@EPEXCH2.qlogic.org> References: <99863D2ED484D449811D97A4C44C9CBD82FAA5@EPEXCH2.qlogic.org> <487CC4B8.5080506@mellanox.co.il> <99863D2ED484D449811D97A4C44C9CBD82FAF0@EPEXCH2.qlogic.org> <5E701717F2B2ED4EA60F87C8AA57B7CC083118DD@venom2> <99863D2ED484D449811D97A4C44C9CBD82FB9D@EPEXCH2.qlogic.org> Message-ID: <20080716124743.6bd8634c.weiny2@llnl.gov> You need to run set_nodedesc.sh right after the hardware driver is loaded. We don't use openibd here so I don't know exactly what it does. Our modprobe.conf runs set_nodedesc.sh as part of the "install" command for the hardware so the node descriptor is set before the node starts responding to MAD's. So far this has worked for us. Ira On Wed, 16 Jul 2008 13:36:02 -0500 "John Russo" wrote: > It looks like this issue is important to a few of us. Can anyone > comment on these concerns? > > -----Original Message----- > From: Chien Tung [mailto:ctung at NetEffect.com] > Sent: Tuesday, July 15, 2008 4:32 PM > To: John Russo; vlad at mellanox.co.il > Cc: general at lists.openfabrics.org > Subject: RE: [ofa-general] Question > > > > We have found that causes openibd to be started before > > networking and therefore the NodeDescription, when returned > > from the SM, does not always contain the hostname of the > > system when ibhosts is run. > > > > Is this a problem for anyone else? > > We have a reported issue on RHEL4U4 with device file not getting created > properly. We believe it is related to this change but have not fully > root cause the problem. > > Chien > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From michael.heinz at qlogic.com Wed Jul 16 14:10:19 2008 From: michael.heinz at qlogic.com (Mike Heinz) Date: Wed, 16 Jul 2008 16:10:19 -0500 Subject: [ofa-general] [PATCH] mvapich-1.0.1-2533 Message-ID: The mpivars.sh and mpivars.csh scripts do not set the LD_LIBRARY_PATH, which means that mvapich programs may not run unless the path is explicitly set. In addition, mpivars.csh has a bug where it does not set the path, either. I will be submitting a similar patch for mvapich2 shortly. --- mvapich.spec.orig 2008-07-16 17:06:44.000000000 -0400 +++ mvapich.spec 2008-07-16 16:49:27.000000000 -0400 @@ -300,17 +300,25 @@ if ! echo \${PATH} | grep -q %{_prefix}/bin ; then export PATH=%{_prefix}/bin:\${PATH} fi +if ! echo \${LD_LIBRARY_PATH} | grep -q %{_prefix}/lib ; then + export LD_LIBRARY_PATH=%{_prefix}/lib:%{_prefix}/lib:/shared:\${LD_LIBRARY_PATH } +fi EOF # Script for csh cat < %{build_root}/%{_prefix}/bin/%{shell_scripts_basename}.csh -if (?$path) then - if ( "\${path}" !~ *%{_prefix}/bin* ) then - setenv path %{_prefix}/bin:\$path +if ("\$path" !~ *%{_prefix}/bin) then + set path=(%{_prefix}/bin \$path) +endif + +if ("1" == "\$?LD_LIBRARY_PATH") then + if ("\$LD_LIBRARY_PATH" !~ *%{_prefix}/lib) then + setenv LD_LIBRARY_PATH %{_prefix}/lib:%{_prefix}/lib/shared:\${LD_LIBRARY_PATH} endif else - setenv path %{_prefix}/bin: + setenv LD_LIBRARY_PATH %{_prefix}/lib:%{_prefix}/lib/shared endif + EOF -- Michael Heinz Principal Engineer, Qlogic Corporation King of Prussia, Pennsylvania -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralph.campbell at qlogic.com Wed Jul 16 15:33:13 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Wed, 16 Jul 2008 15:33:13 -0700 Subject: [ofa-general] [PATCH] IB/core - ib_sa_remove_one() can panic due to NULL pointer dereference Message-ID: <20080716223312.21857.21231.stgit@eng-46.mv.qlogic.com> If an HCA does not get a SM LID, ib_sa_remove_one() can cause a NULL pointer bug when it calls kref_put(). This is easy to see since ib_sa_add_one() sets sm_ah to NULL. Signed-off-by: Ralph Campbell --- drivers/infiniband/core/sa_query.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index 1341de7..7863a50 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -1064,7 +1064,8 @@ static void ib_sa_remove_one(struct ib_device *device) for (i = 0; i <= sa_dev->end_port - sa_dev->start_port; ++i) { ib_unregister_mad_agent(sa_dev->port[i].agent); - kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah); + if (sa_dev->port[i].sm_ah) + kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah); } kfree(sa_dev); From Nathan.Dauchy at noaa.gov Wed Jul 16 16:46:51 2008 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Wed, 16 Jul 2008 17:46:51 -0600 Subject: [ofa-general] OpenSM "Dead end on path to LID" Message-ID: <487E886B.8010906@noaa.gov> Greetings, We have recently expanded our Infiniband tree and are running into problems when all hosts are booted. Details are below. Please let me know if there is a more appropriate forum for this issue. Thanks! With less than 600 hosts, everything seems to be working fine. With more than 650 or so, we start seeing the following symptoms: # ibdiagnet -o . -lw 4x -pc -I- Discovering ... 721 nodes (68 Switches & 653 CA-s) discovered. ... -I--------------------------------------------------- -I- PM Counters Info -I--------------------------------------------------- -E- Could not get PM info: "pmGetPortCounters 0x0139 1" failed 4 consecutive times. -E- Could not get PM info: "pmGetPortCounters 0x0139 4" failed 4 consecutive times. There are 29 of those "Could not get PM info" errors. Basic IB communication still works at this point, but after restarting the subnet manager, ping via IPoIB stops working between some of the switches, and a LOT of messages like the following show up in osm.log: Jul 16 22:32:13 795167 [41E02940] 0x01 -> __osm_pr_rcv_get_path_parms: ERR 1F07: Dead end on path to LID 0x9 from switch for GUID 0x000002c900000023 Jul 16 22:36:04 895497 [45007940] 0x01 -> __osm_pr_rcv_get_path_parms: ERR 1F07: Dead end on path to LID 0x5D7 from switch for GUID 0x000002c900000052 I have tried modifying "opensm.conf" to include: LMC=0 (was 2) TIMEOUT=500 (was 200) but that did not seem to help. Subnet manager host is running CentOS-5.1, kernel 2.6.18-53.1.21.el5, OFED-1.3.1, OpenSM 3.1.11 Hosts are running either RHEL-4.4, kernel 2.6.20.20, OFED-1.2.5.1 CentOS-5.1, kernel 2.6.22.19, OFED-1.3.1 Storage vendor OS based on CentOS, kernel 2.6.9-42.0.10.ELsmp, OFED-1.2.5.1 Can anyone suggest a fix or other diagnostics we can run to help narrow down the problem? -Nathan From prescott at hpc.ufl.edu Wed Jul 16 17:34:27 2008 From: prescott at hpc.ufl.edu (Craig Prescott) Date: Wed, 16 Jul 2008 20:34:27 -0400 Subject: [ofa-general] "ibdiagnet -r" and zero systemguids Message-ID: <487E9393.1080202@hpc.ufl.edu> Hi; When we run 'ibdiagnet -r' on our OFED 1.2 cluster, it bombs with a complaint about a system guid that is zero on our only PCI-X HCA in the fabric (see appended). ibdiagnet seems to be trying to saw off the leading zeroes from the system guid, and to have nothing left afterwards seems unexpected. Running 'ibdiagnet -r' from an OFED 1.3.1 machine does not bomb, but I am still concerned/unclear. My questions are: is it ok to have an HCA running around on your fabric with a system guid of zero? What if there was more than one? Is there any way to assign this HCA a sensible system guid, and would it be useful? The HCA in question is a Cougar cub running the 3.5.0 firmware from Mellanox. FWIW, the node and port guids for this HCA look sensible: [root at submit ~]# tvflash -g HCA #0 Node GUID = 0005ad0000050948 Port1 GUID = 0005ad0000050949 Port2 GUID = 0005ad000005094a If it isn't obvious already, I confess I'm not clear about how system guids are used. From what I can gather from google-ing around, a system guid of zero for an HCA means that the HCA vendor simply did not assign one. I am under the impression that this is uncommon, but not unheard of. Is that correct? I did some searches through both volumes of the 1.2.1 IB spec and came up empty, but I could have easily missed any substantial discussion about system guids. Any pointers or enlightenment in this area would be appreciated. Thanks, Craig Prescott UF HPC Center [root at submit ~]# ibdiagnet -r Loading IBDIAGNET from: /usr/lib64/ibdiagnet1.2 Loading IBDM from: /usr/lib64/ibdm1.2 -W- Topology file is not specified. Reports regarding cluster links will use direct routes. -I- Using port 1 as the local port. -I- Discovering the subnet ... 394 nodes (46 Switches & 348 CA-s) discovered. -I- Parsing Subnet file:/tmp/ibdiagnet.lst -I- Defined 382/394 systems/nodes -I--------------------------------------------------- -I- Bad Guids Info -I--------------------------------------------------- -W- Found Device with SystemGUID=0x0000000000000000: a HCA The Local Device "submit.ufhpc/P1" PortGUID=0x0005ad0000050949 at direct path="" ... -I--------------------------------------------------- -I- mgid-mlid-HCAs matching table -I--------------------------------------------------- mgid | mlid | HCAs -------------------------------------------------------------------------------- ERROR can't use empty string as operand of "+" while executing "if {([removeLeadingZeros $n] > [removeLeadingZeros $end] + 1)} { if {$start == $end} { append res "$end," } else { ..." (procedure "groupNumRanges" line 15) invoked from within "groupNumRanges $NEW_GROUPS($pNs)" (procedure "groupingEngine" line 24) invoked from within "groupingEngine $groups" (procedure "compressNames" line 12) invoked from within "compressNames $mlidHcas" (procedure "reportFabQualities" line 82) invoked from within "reportFabQualities" can't use empty string as operand of "+" From prescott at hpc.ufl.edu Wed Jul 16 17:41:12 2008 From: prescott at hpc.ufl.edu (Craig Prescott) Date: Wed, 16 Jul 2008 20:41:12 -0400 Subject: [ofa-general] Re: "ibdiagnet -r" and zero systemguids In-Reply-To: <487E9393.1080202@hpc.ufl.edu> References: <487E9393.1080202@hpc.ufl.edu> Message-ID: <487E9528.4000202@hpc.ufl.edu> I forgot to add that other than this SystemGUID=0x0000000000000000 issue, the HCA appears to work perfectly. Thanks, Craig Craig Prescott wrote: > > Hi; > > When we run 'ibdiagnet -r' on our OFED 1.2 cluster, > it bombs with a complaint about a system guid that is > zero on our only PCI-X HCA in the fabric (see appended). > ibdiagnet seems to be trying to saw off the leading zeroes > from the system guid, and to have nothing left afterwards > seems unexpected. > > Running 'ibdiagnet -r' from an OFED 1.3.1 machine does > not bomb, but I am still concerned/unclear. > > My questions are: is it ok to have an HCA running > around on your fabric with a system guid of zero? > What if there was more than one? Is there any way to > assign this HCA a sensible system guid, and would it > be useful? > > The HCA in question is a Cougar cub running the 3.5.0 > firmware from Mellanox. FWIW, the node and port guids > for this HCA look sensible: > > [root at submit ~]# tvflash -g > HCA #0 > Node GUID = 0005ad0000050948 > Port1 GUID = 0005ad0000050949 > Port2 GUID = 0005ad000005094a > > If it isn't obvious already, I confess I'm not clear > about how system guids are used. From what I can gather > from google-ing around, a system guid of zero for an HCA > means that the HCA vendor simply did not assign one. I > am under the impression that this is uncommon, but not > unheard of. Is that correct? > > I did some searches through both volumes of the 1.2.1 IB > spec and came up empty, but I could have easily missed any > substantial discussion about system guids. Any pointers or > enlightenment in this area would be appreciated. > > Thanks, > Craig Prescott > UF HPC Center > > [root at submit ~]# ibdiagnet -r > Loading IBDIAGNET from: /usr/lib64/ibdiagnet1.2 > Loading IBDM from: /usr/lib64/ibdm1.2 > -W- Topology file is not specified. > Reports regarding cluster links will use direct routes. > -I- Using port 1 as the local port. > -I- Discovering the subnet ... 394 nodes (46 Switches & 348 CA-s) > discovered. > > -I- Parsing Subnet file:/tmp/ibdiagnet.lst > -I- Defined 382/394 systems/nodes > > -I--------------------------------------------------- > -I- Bad Guids Info > -I--------------------------------------------------- > -W- Found Device with SystemGUID=0x0000000000000000: > a HCA The Local Device "submit.ufhpc/P1" > PortGUID=0x0005ad0000050949 at direct path="" > ... > -I--------------------------------------------------- > -I- mgid-mlid-HCAs matching table > -I--------------------------------------------------- > mgid | mlid | HCAs > -------------------------------------------------------------------------------- > > > > ERROR can't use empty string as operand of "+" > while executing > "if {([removeLeadingZeros $n] > [removeLeadingZeros $end] + 1)} { > if {$start == $end} { > append res "$end," > } else { > ..." > (procedure "groupNumRanges" line 15) > invoked from within > "groupNumRanges $NEW_GROUPS($pNs)" > (procedure "groupingEngine" line 24) > invoked from within > "groupingEngine $groups" > (procedure "compressNames" line 12) > invoked from within > "compressNames $mlidHcas" > (procedure "reportFabQualities" line 82) > invoked from within > "reportFabQualities" can't use empty string as operand of "+" > > From rdreier at cisco.com Wed Jul 16 17:46:50 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 16 Jul 2008 17:46:50 -0700 Subject: [ofa-general] [PATCH] IB/core - ib_sa_remove_one() can panic due to NULL pointer dereference In-Reply-To: <20080716223312.21857.21231.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Wed, 16 Jul 2008 15:33:13 -0700") References: <20080716223312.21857.21231.stgit@eng-46.mv.qlogic.com> Message-ID: > If an HCA does not get a SM LID, ib_sa_remove_one() can cause a NULL > pointer bug when it calls kref_put(). This is easy to see since > ib_sa_add_one() sets sm_ah to NULL. Have you seen this in practice? Because ib_sa_add_one() calls update_sm_ah() before returning, so ib_sa_remove_one() shouldn't see a NULL sm_ah because of that. And ib_sa_event() schedules the update task after setting sm_ah to NULL, and ib_sa_remove_one() does flush_scheduled_work() so update_sm_ah() should be guaranteed to run in that case too. I do see the possibility of hitting one of the failures in update_sm_ah() and ending up not setting sm_ah, so your patch is correct, but I'm just wondering how you hit this in practice. I also see what I guess is techically a bug in ib_sa_add_one(), namely that it does ib_set_client_data(device, &sa_client, sa_dev); before INIT_IB_EVENT_HANDLER(&sa_dev->event_handler, device, ib_sa_event); if (ib_register_event_handler(&sa_dev->event_handler)) goto err; and if that function returns an error, then sa_dev gets freed and ib_sa_remove_one() ends up doing a use-after-free. However with the current code, ib_register_event_handler() can never return an error. I wonder if the best fix is just to make ib_register_event_handler() a void function. - R. From Nathan.Dauchy at noaa.gov Wed Jul 16 18:35:08 2008 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Wed, 16 Jul 2008 19:35:08 -0600 Subject: [ofa-general] OpenSM "Dead end on path to LID" In-Reply-To: <487E886B.8010906@noaa.gov> References: <487E886B.8010906@noaa.gov> Message-ID: <487EA1CC.7060308@noaa.gov> Nathan Dauchy wrote: > Greetings, > > We have recently expanded our Infiniband tree and are running into > problems when all hosts are booted. Details are below. Please let me > know if there is a more appropriate forum for this issue. Thanks! > > > With less than 600 hosts, everything seems to be working fine. With > more than 650 or so, we start seeing the following symptoms: > > # ibdiagnet -o . -lw 4x -pc > -I- Discovering ... 721 nodes (68 Switches & 653 CA-s) discovered. > ... > -I--------------------------------------------------- > -I- PM Counters Info > -I--------------------------------------------------- > -E- Could not get PM info: > "pmGetPortCounters 0x0139 1" failed 4 consecutive times. > -E- Could not get PM info: > "pmGetPortCounters 0x0139 4" failed 4 consecutive times. > > There are 29 of those "Could not get PM info" errors. > > Basic IB communication still works at this point, but after restarting > the subnet manager, ping via IPoIB stops working between some of the > switches, and a LOT of messages like the following show up in osm.log: > > Jul 16 22:32:13 795167 [41E02940] 0x01 -> __osm_pr_rcv_get_path_parms: > ERR 1F07: Dead end on path to LID 0x9 from switch for GUID > 0x000002c900000023 > Jul 16 22:36:04 895497 [45007940] 0x01 -> __osm_pr_rcv_get_path_parms: > ERR 1F07: Dead end on path to LID 0x5D7 from switch for GUID > 0x000002c900000052 > Looking through osm.log a bit more, I also found a handful of errors like these: Jul 17 01:31:29 345329 [46E0A940] 0x01 -> __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for node 0x000002c900000048(MT47396 Infiniscale-III Mellanox Technologies) port 14. Adding to light sweep sampling list Jul 17 01:31:29 345340 [46E0A940] 0x01 -> Directed Path Dump of 4 hop path: Path = 0,1,20,7,15 Jul 17 01:31:29 345381 [46E0A940] 0x01 -> __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for node 0x000002c900000049(MT47396 Infiniscale-III Mellanox Technologies) port 15. Adding to light sweep sampling list Jul 17 01:31:29 345390 [46E0A940] 0x01 -> Directed Path Dump of 3 hop path: Path = 0,1,22,11 Does that indicate a problem as well? Thanks, Nathan From bobzankoski at sympatico.ca Wed Jul 16 18:56:08 2008 From: bobzankoski at sympatico.ca (Mr. Micheal Anderson) Date: Wed, 16 Jul 2008 21:56:08 -0400 Subject: [ofa-general] *** Lucky Winner Message-ID: <20080717015618.VKNI1531.tomts23-srv.bellnexxia.net@toip40-bus.srvr.bell.ca> SWISS ON-LINE LOTTERY P O Box 1010 Liverpool, L70 1NL UNITED KINGDOM REF NO: SL/74/368/05 BATCH NO:SL-121-LT-11-12-05 You won the sum of £1,500 000.00 GBP from our monthly PROGRAM, held on the 17th July 2008.You are adviced to Contact Mr.Michael Anderson to claim your prize. Provide him with your 1,Names: 2,Address: 3,Age: 4,Occupation, 5,Tel: 6,Country: Contact Email:michealanderson at rocketmail.com Sincerely, Mr. Micheal Anderson.(Swiss Lottery Online Co-ordinator) From bobzankoski at sympatico.ca Wed Jul 16 18:56:08 2008 From: bobzankoski at sympatico.ca (Mr. Micheal Anderson) Date: Wed, 16 Jul 2008 21:56:08 -0400 Subject: [ofa-general] *** Lucky Winner Message-ID: <20080717015626.TDUZ1589.tomts52-srv.bellnexxia.net@toip41-bus.srvr.bell.ca> SWISS ON-LINE LOTTERY P O Box 1010 Liverpool, L70 1NL UNITED KINGDOM REF NO: SL/74/368/05 BATCH NO:SL-121-LT-11-12-05 You won the sum of £1,500 000.00 GBP from our monthly PROGRAM, held on the 17th July 2008.You are adviced to Contact Mr.Michael Anderson to claim your prize. Provide him with your 1,Names: 2,Address: 3,Age: 4,Occupation, 5,Tel: 6,Country: Contact Email:michealanderson at rocketmail.com Sincerely, Mr. Micheal Anderson.(Swiss Lottery Online Co-ordinator) From jsquyres at cisco.com Wed Jul 16 19:10:24 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Wed, 16 Jul 2008 22:10:24 -0400 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: <487C816C.7040301@mellanox.co.il> References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> <487B8727.20003@opengridcomputing.com> <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> <487C816C.7040301@mellanox.co.il> Message-ID: On Jul 15, 2008, at 6:52 AM, Tziporet Koren wrote: >> - What is the proposed release schedule for OFED 1.4? > Alpha - next week and Beta by end of July. Do you really mean alpha one week and beta the next? Is there a target final release date for OFED v1.4? -- Jeff Squyres Cisco Systems From Mayuresh.Kulkarni at morganstanley.com Wed Jul 16 19:12:42 2008 From: Mayuresh.Kulkarni at morganstanley.com (Kulkarni, Mayuresh (IT)) Date: Wed, 16 Jul 2008 22:12:42 -0400 Subject: [ofa-general] libsdp usage In-Reply-To: <200807161811.11145.jackm@dev.mellanox.co.il> References: <200807151745.53152.jackm@dev.mellanox.co.il><15ddcffd0807151216h3fe364bcx5d164ffa15694117@mail.gmail.com> <200807161811.11145.jackm@dev.mellanox.co.il> Message-ID: <1F8E762E1B3F814F9630D0D32A1C65F814A90EF5@NYWEXMB81.msad.ms.com> Hello, We are just starting out with OFED and I am not sure how to troubleshoot the following problem. The setup has two machines, each has a RNIC and they communicate over TCP/IP. From my understanding, just preloading libsdp.so in a TCP/IP sockets based app should work fine. Is that correct? It is not working for a very simple client/server app. I changed the logging level in the libsdp.conf file down to 1. Then I ran the client and server on different machines. When the server and client are both running with libsdp.so preloaded, the logs are appended at the end. The last thing the client says is "connected SDP fd:8 to:192.168.0.50 port 5023" (The IP address belongs to the server's RNIC). But then it just hangs instead of sending some data as it is supposed to. The server logs seem to say that it is just waiting around for something to happen. It is possible I am doing something entirely silly or OFED is not installed properly, so any pointers on what might be happening are greatly appreciated. Thanks a lot, Mayuresh. -------------- Server: Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CONFIG: use both server * *:* Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CONFIG: use both client * *:* Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp Max file descriptors:1024 Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: result <-1> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: result <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <1:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: result <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: result <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <1:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp CLOSE: result <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp IOCTL: request <21537> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp IOCTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp SOCKET: domain <2> type <1> protocol <0> Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp MATCH PROGRAM: => both Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp SOCKET: making TCP socket Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp SOCKET: making SDP shadow socket type:1 proto:6 Wed Jul 16 21:24:13 2008 echoServer[1999] libsdp SOCKET: Wed Client: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONFIG: use both server * *:* Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONFIG: use both client * *:* Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp Max file descriptors:1024 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: result <-1> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: result <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <1:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: result <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: result <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <1> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <1:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CLOSE: result <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp IOCTL: request <21537> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp IOCTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <2> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SOCKET: domain <2> type <1> protocol <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp MATCH PROGRAM: => both Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SOCKET: making TCP socket Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SOCKET: making SDP shadow socket type:1 proto:6 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SOCKET: Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: level <1> name <2> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <5> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <-1:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <5> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <-1:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp GETSOCKNAME Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp GETSOCKNAME: address is:0.0.0.0 port:0 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp GETSOCKNAME result <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <5> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <-1:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp BIND: type <2> IP <192.168.0.100> port <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp __sdp_sockaddr_to_sdp: Given IPv4 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: starting search for common free port Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: taking loop (1) Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: creating the two sockets Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: binding first tcp socket Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: listening on first socket Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: first socket port:33035 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: binding second socket Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp find_free_port: return port:<33035> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp close_and_bind: closing <11> binding <8> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp close_and_bind: returning <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp close_and_bind: closing <10> binding <9> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp close_and_bind: returning <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp BIND: result <0:-1> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp IOCTL: request <21537> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp IOCTL: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: level <1> name <9> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: substitute level 27 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: level <6> name <1> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp SETSOCKOPT: result <0:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: command <5> argument <0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp FCNTL: result <-1:0> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONNECT: domain <2> IP <192.168.0.50> port <5023> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONNECT: fd <8> opts are <0x802> Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp MATCH: matching 192.168.0.50:5023 to use both * *:* => Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp POSITIVE Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp MATCH CONNECT: => both Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONNECT: libsdp does not support async connect in BOTH, moving to SDP only Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp __sdp_sockaddr_to_sdp: Given IPv4 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONNECT: connecting through SDP Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONNECT: connected SDP fd:8 to:192.168.0.50 port 5023 Wed Jul 16 21:24:22 2008 echoClient[2309] libsdp CONNECT: References: Message-ID: <487EF563.9090102@voltaire.com> Roland Dreier wrote: > ... 2.6.26 is already out, so it's time to review what my plans are for the merge window opens. > > Core: > > - I'm waiting to merge the RDMA_CM_EVENT_ADDR_CHANGE changes that > depend on core networking changes until such changes are upstream. > Or, please remind me when that happens. Dave, There are some rdma patches for 2.6.27 which are dependent on few networking patches (below), so Roland can't push them before the net changes gets in, such that enough time has to be left for the rdma push before the merge window closes... Or. > commit 01f3109de49a889db8adf9116449727547ee497e > Author: Or Gerlitz > Date: Fri Jun 13 18:12:02 2008 -0700 > > bonding: deliver netdev event for fail-over under the active-backup mode > > commit 709f8a45e8521f2f4229e5fdf3ded1fb77e2ca4e > Author: Or Gerlitz > Date: Fri Jun 13 18:12:01 2008 -0700 > > bonding: bond_change_active_slave() cleanup under active-backup > > commit c1da4ac752b8b0411791d26c678fcf23d2eed242 > Author: Or Gerlitz > Date: Fri Jun 13 18:12:00 2008 -0700 > > net/core: add NETDEV_BONDING_FAILOVER event From davem at davemloft.net Thu Jul 17 00:37:13 2008 From: davem at davemloft.net (David Miller) Date: Thu, 17 Jul 2008 00:37:13 -0700 (PDT) Subject: [ofa-general] Re: InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: <487EF563.9090102@voltaire.com> References: <487EF563.9090102@voltaire.com> Message-ID: <20080717.003713.193704510.davem@davemloft.net> From: Or Gerlitz Date: Thu, 17 Jul 2008 10:31:47 +0300 > There are some rdma patches for 2.6.27 which are dependent on few > networking patches (below), so Roland can't push them before the net > changes gets in, such that enough time has to be left for the rdma push > before the merge window closes... I'm pushing to Linus this coming Sunday. I wanted to do it sooner but my tree isn't in a state where I can merge it just yet, and Linus is going away for 3 days starting Thursday. I'm sorry if this makes things difficult, but it's the best I can do at the moment. From ogerlitz at voltaire.com Thu Jul 17 00:44:55 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 17 Jul 2008 10:44:55 +0300 Subject: [ofa-general] Re: InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: <20080717.003713.193704510.davem@davemloft.net> References: <487EF563.9090102@voltaire.com> <20080717.003713.193704510.davem@davemloft.net> Message-ID: <487EF877.3050803@voltaire.com> David Miller wrote: > I'm pushing to Linus this coming Sunday. I'm sorry if this makes things difficult, but it's the best I can do at the moment. > thanks for the update, I believe this will leave Roland enough time to do his push, Or. From ogerlitz at voltaire.com Thu Jul 17 00:47:43 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 17 Jul 2008 10:47:43 +0300 Subject: [ofa-general][PATCH v2 1/2]mlx4: Multiple completion vectors support In-Reply-To: References: <483C2966.1080606@mellanox.co.il> <48440414.4080909@mellanox.co.il> <4844E40A.9080209@mellanox.co.il> <48511089.8050609@voltaire.com> <48512C94.1040306@voltaire.com> Message-ID: <487EF91F.5030706@voltaire.com> Roland Dreier wrote: > Well, both of your statements seem to be true: nothing sets the affinity for the interrupts created for multiple EQs, and there is not any simply way to guarantee that CQ vector 5 is sent to CPU 5 that I see. Going back to this, looking in Documentation/IRQ-affinity.txt my understanding is that interrupt affinity can be set manually through writing to /proc/irq/IRQ#/smp_affinity so the admin can do that. Or. From tziporet at dev.mellanox.co.il Thu Jul 17 02:03:13 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 17 Jul 2008 12:03:13 +0300 Subject: [ofa-general] ***SPAM*** InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: References: <487B820E.4020002@mellanox.co.il> <15ddcffd0807151220m37204055rf6890037b445bd34@mail.gmail.com> Message-ID: <487F0AD1.6060200@mellanox.co.il> Roland Dreier wrote: > Right: in the test > > + if ((dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_LOCAL_INV) && > + (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_REMOTE_INV) && > + (dev->dev->caps.bmme_flags & MLX4_BMME_FLAG_FAST_REG_WR)) > + props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; > > you will see that not all the bmme_flags required are set by FW 2.5. > > If someone is interested to get a FW that implement is please reply to me We should have such a FW next week Tziporet From tziporet at dev.mellanox.co.il Thu Jul 17 02:33:46 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 17 Jul 2008 12:33:46 +0300 Subject: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008) In-Reply-To: References: <5D49E7A8952DC44FB38C38FA0D758EAD14EB23@mtlexch01.mtl.com> <487B8727.20003@opengridcomputing.com> <5AA159FB-F6DB-4672-B33B-4ECFCC921A0A@cisco.com> <487C816C.7040301@mellanox.co.il> Message-ID: <487F11FA.6060709@mellanox.co.il> Jeff Squyres wrote: > On Jul 15, 2008, at 6:52 AM, Tziporet Koren wrote: > >>> - What is the proposed release schedule for OFED 1.4? >> Alpha - next week and Beta by end of July. > > Do you really mean alpha one week and beta the next? Well - alpha should be this week but not all components was ready > > Is there a target final release date for OFED v1.4? > October 6 All milestones dates are listed at: http://www.openfabrics.org/txt/woody/roadmap.txt Tziporet From vlad at lists.openfabrics.org Thu Jul 17 02:58:14 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Thu, 17 Jul 2008 02:58:14 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080717-0200 daily build status Message-ID: <20080717095814.617A4E28002@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Failed: Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080717-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From eli at dev.mellanox.co.il Thu Jul 17 03:46:33 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 17 Jul 2008 13:46:33 +0300 Subject: [ofa-general] Infinband and DRBD In-Reply-To: References: <2f3bf9a60807122324j60e48a9gddf3b01612c20d0b@mail.gmail.com> Message-ID: <20080717104633.GA9308@mtls03> > I think it answers my question. I'm curious to know IPOIB-CM would offer > better performance but can't find much information on the difference > between IPOIB and IPOIB-CM on Google. Does anyone have a link or a > little bit of info on the differences? > IPoIB CM uses RC transport service to send/receive network traffic while UD mode uses UD transport service. Generally, CM mode will give you better throughput and UD mode will give better latency but UD mode has gone through a lot of improvements and gives decent throughput too. From ogerlitz at voltaire.com Thu Jul 17 04:37:17 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 17 Jul 2008 14:37:17 +0300 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support In-Reply-To: <20080716150125.GE7921@mtls03> References: <20080715161251.GA16299@mtls03> <20080716065746.GA720@mtls03> <487DD7A0.4040709@voltaire.com> <20080716150125.GE7921@mtls03> Message-ID: <487F2EED.2090108@voltaire.com> Eli Cohen wrote: > But the when I said "without RCA" it was with all the patches applied but RCA was disabled through the module param. So the gain in performance can be attibuted to RCA. This could be a somehow simplified approach, > In tests I made on my machine I got the following results with netperf TCP, 4 streams. > Without RCA: 730 MB/s With RCA: 970 MB/s On my setup, without any of the patches you are using I get 800 MB/s with single netperf TCP, 850 MB/s with two streams and 900 MB/s with three streams, the fourth stream cuts the result by half down to 500 MB/s but I did not try to play with process/irq CPU affinity and/or run some of the streams on different QPs (eg through child interfaces) This suggests that an check should be made on the performance enhancement provided by the multiple EQ patch by itself, and then see if/how much RCA adds on it. For example if for four streams it goes 500 (no patch) --> 750 (Multiple EQ) --> 950 (EQ + RCA) but under tuning the first number jumps to 950 MB/s then there's no actual gain in this setting, thoughts? Or. From anuj01 at gmail.com Thu Jul 17 05:13:17 2008 From: anuj01 at gmail.com (=?UTF-8?B?4KSF4KSo4KWB4KSc?=) Date: Thu, 17 Jul 2008 17:43:17 +0530 Subject: [ofa-general] Hi- "make" error during installation of librdmacm Message-ID: Hi librdmacm/configure script runs successfully . But when executing make then getting error as follows : . . gcc -g -Wall -D_GNU_SOURCE -I /home/anuj/my_ofed/include /home/anuj/my_ofed/lib64 -o examples/.libs/ucmatose cmatose.o -L/home/anuj/librdmacm-1.0.3 ./src/.libs/librdmacm.so /home/anuj/my_ofed/lib64//libibverbs.so -Wl,--rpath -Wl,/home/anuj/my_ofed/lib64/ /home/anuj/my_ofed/lib64: file not recognized: Is a directory collect2: ld returned 1 exit status make[1]: *** [examples/ucmatose] Error 1 make[1]: Leaving directory `/home/anuj/librdmacm-1.0.3' make: *** [all] Error 2 And env. variables set : CFLAGS="-I /home/anuj/my_ofed/include" LDFLAGS="-L /home/anuj/my_ofed/lib64" LD_LIBARAY_PATH="/home/anuj/my_ofed/lib64" is this make file problem or any other?/ please specify. Thanks With Regards Anuj Aggarwal From jackm at dev.mellanox.co.il Thu Jul 17 05:17:00 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 17 Jul 2008 15:17:00 +0300 Subject: [ofa-general] [PATCH V2 for 2.6.27] mlx4: Update/add Mellanox Technologies copyright lines to mlx4 driver files Message-ID: <200807171517.00673.jackm@dev.mellanox.co.il> mlx4: Update Mellanox copyright lines. Update existing Mellanox copyright lines to 2008, and add such lines to files where they are missing. V2: eliminated copyright additions/changes for files Mellanox did not modify. Signed-off-by: Jack Morgenstein --- diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 299f208..4d4a859 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index bcf5064..f766d8d 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index c4cf5b6..71c2a27 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c index 68e9248..5605c56 100644 --- a/drivers/infiniband/hw/mlx4/mr.c +++ b/drivers/infiniband/hw/mlx4/mr.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 89eb6cb..5a83cc3 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c index 12d6bc6..d425652 100644 --- a/drivers/infiniband/hw/mlx4/srq.c +++ b/drivers/infiniband/hw/mlx4/srq.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/mlx4/user.h b/drivers/infiniband/hw/mlx4/user.h index e2d11be..13beede 100644 --- a/drivers/infiniband/hw/mlx4/user.h +++ b/drivers/infiniband/hw/mlx4/user.h @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/alloc.c b/drivers/net/mlx4/alloc.c index f9d6b4d..096bca5 100644 --- a/drivers/net/mlx4/alloc.c +++ b/drivers/net/mlx4/alloc.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/catas.c b/drivers/net/mlx4/catas.c index aa95287..f094ee0 100644 --- a/drivers/net/mlx4/catas.c +++ b/drivers/net/mlx4/catas.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c index 70dff94..bc40331 100644 --- a/drivers/net/mlx4/cmd.c +++ b/drivers/net/mlx4/cmd.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 95e87a2..9bb50e3 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -2,7 +2,7 @@ * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2004 Voltaire, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c index e141a15..6d61dab 100644 --- a/drivers/net/mlx4/eq.c +++ b/drivers/net/mlx4/eq.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index 2b5006b..8f1b61e 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index a0e046c..c548ead 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -1,6 +1,6 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c index 2a5bef6..baf4bf6 100644 --- a/drivers/net/mlx4/icm.c +++ b/drivers/net/mlx4/icm.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/icm.h b/drivers/net/mlx4/icm.h index 6c44edf..ab56a2f 100644 --- a/drivers/net/mlx4/icm.h +++ b/drivers/net/mlx4/icm.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/intf.c b/drivers/net/mlx4/intf.c index 4a6c4d5..0e7eb10 100644 --- a/drivers/net/mlx4/intf.c +++ b/drivers/net/mlx4/intf.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index d373601..af2debc 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -1,7 +1,7 @@ /* * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/mcg.c b/drivers/net/mlx4/mcg.c index b4b5787..c83f88c 100644 --- a/drivers/net/mlx4/mcg.c +++ b/drivers/net/mlx4/mcg.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a4023c2..2253d41 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -2,7 +2,7 @@ * Copyright (c) 2004, 2005 Topspin Communications. All rights reserved. * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2004 Voltaire, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 03a9abc..f93ff35 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004 Topspin Communications. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c index ee5484c..c49a860 100644 --- a/drivers/net/mlx4/qp.c +++ b/drivers/net/mlx4/qp.c @@ -1,7 +1,7 @@ /* * Copyright (c) 2004 Topspin Communications. All rights reserved. * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc. All rights reserved. - * Copyright (c) 2005 Mellanox Technologies. All rights reserved. + * Copyright (c) 2005, 2006, 2007, 2008 Mellanox Technologies. All rights reserved. * Copyright (c) 2004 Voltaire, Inc. All rights reserved. * * This software is available to you under a choice of one of two diff --git a/drivers/net/mlx4/reset.c b/drivers/net/mlx4/reset.c index e199715..3951b88 100644 --- a/drivers/net/mlx4/reset.c +++ b/drivers/net/mlx4/reset.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/mlx4/srq.c b/drivers/net/mlx4/srq.c index d23f46d..533eb6d 100644 --- a/drivers/net/mlx4/srq.c +++ b/drivers/net/mlx4/srq.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2006, 2007 Cisco Systems, Inc. All rights reserved. + * Copyright (c) 2007, 2008 Mellanox Technologies. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU From karinas9 at seanthomas.com Thu Jul 17 05:21:46 2008 From: karinas9 at seanthomas.com (Abigail Hutton) Date: Thu, 17 Jul 2008 20:21:46 +0800 Subject: [ofa-general] InternationalPharmacyFDAapprovedWorldwideShipping Message-ID: <01c8e84a$bc092900$af181074@karinas9> CanadianThankYouWorldwideShipping http://ifbuqw.bay.livefilestore.com/y1pQc2-sWmMUW_UiJmTpF2rkNn7G17zeSh5GmJjB8NJos-brZjMOV_8T-HjbzeBytHbv7d-VXbkFysUaYfmZOARdA -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at dev.mellanox.co.il Thu Jul 17 06:02:11 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 17 Jul 2008 16:02:11 +0300 Subject: [ofa-general] [PATCH 0/3] Recieve Core Affinity support In-Reply-To: <487F2EED.2090108@voltaire.com> References: <20080715161251.GA16299@mtls03> <20080716065746.GA720@mtls03> <487DD7A0.4040709@voltaire.com> <20080716150125.GE7921@mtls03> <487F2EED.2090108@voltaire.com> Message-ID: <20080717130211.GB9308@mtls03> On Thu, Jul 17, 2008 at 02:37:17PM +0300, Or Gerlitz wrote: > Eli Cohen wrote: > On my setup, without any of the patches you are using I get 800 MB/s with > single netperf TCP, 850 MB/s with two streams and 900 MB/s with three > streams, the fourth stream cuts the result by half down to 500 MB/s but I > did not try to play with process/irq CPU affinity and/or run some of the > streams on different QPs (eg through child interfaces) If you're seeking improvements through using child interfaces or playing with irq affinity, that's legitimate but it does not suit everyone's needs. > > This suggests that an check should be made on the performance enhancement > provided by the multiple EQ patch by itself, and then see if/how much RCA > adds on it. For example if for four streams it goes > > 500 (no patch) --> 750 (Multiple EQ) --> 950 (EQ + RCA) but under tuning > the first number jumps to 950 MB/s then there's no actual gain in this > setting, thoughts? > Multiple EQ by itself won't buy you anything unless the application (IPoIB in this case) uses it. From eli at mellanox.co.il Thu Jul 17 06:14:34 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Thu, 17 Jul 2008 16:14:34 +0300 Subject: [ofa-general] [PATCH v2] IB/ipoib: Add Receive Core Affinity (RCA) support Message-ID: <20080717131434.GA31548@mtls03> The patch adds RCA support for datagram mode. It creates a number of additional receive rings, equal to the number of CPU cores, and the counts on the hardware to distribute the received packets based on their IP address and port number; packets that are neither TCP nor UDP, are routed to the regular UD QP. It does some arrangements in the receive related data structs so as to unify processing of received packets to the regular UD QP or to the any of the RCA rings. The receive flow uses napi to provide context for simultaneous handling of received packets. Signed-off-by: Eli Cohen --- Changes: Fix CM mode (previous patch ruined it) remove debug print drivers/infiniband/ulp/ipoib/ipoib.h | 35 ++- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 + drivers/infiniband/ulp/ipoib/ipoib_ethtool.c | 63 ++++- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 202 +++++++++++--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 89 +++++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 378 +++++++++++++++++++++++++- 6 files changed, 689 insertions(+), 86 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index b0ffc9a..bb6534d 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -105,6 +105,8 @@ enum { MAX_SEND_CQE = 16, IPOIB_CM_COPYBREAK = 256, + + MAX_RCA_QPS = 8, }; #define IPOIB_OP_RECV (1ul << 31) @@ -267,6 +269,26 @@ struct ipoib_lro { struct net_lro_desc lro_desc[IPOIB_MAX_LRO_DESCRIPTORS]; }; +struct rca_ring_ctx { + struct ib_qp *qp; + struct ib_cq *cq; + struct ib_wc ibwc[IPOIB_NUM_WC]; + struct napi_struct napi; + struct ipoib_rx_buf *rx_ring; + struct ib_recv_wr rx_wr; + struct ib_sge rx_sge[IPOIB_UD_RX_SG]; + struct ipoib_lro lro; + unsigned long rx_packets; + unsigned long rx_dropped; + unsigned long rx_bytes; +}; + +struct ipoib_rca { + struct ib_qp *rca_qp; + int nrings; + struct rca_ring_ctx ring_arr[0]; +}; + /* * Device private locking: tx_lock protects members used in TX fast * path (and we use LLTX so upper layers don't do extra locking). @@ -317,7 +339,7 @@ struct ipoib_dev_priv { unsigned int mcast_mtu; unsigned int max_ib_mtu; - struct ipoib_rx_buf *rx_ring; + struct rca_ring_ctx rx_ring; spinlock_t tx_lock; struct ipoib_tx_buf *tx_ring; @@ -328,11 +350,6 @@ struct ipoib_dev_priv { unsigned tx_outstanding; struct ib_wc send_wc[MAX_SEND_CQE]; - struct ib_recv_wr rx_wr; - struct ib_sge rx_sge[IPOIB_UD_RX_SG]; - - struct ib_wc ibwc[IPOIB_NUM_WC]; - struct list_head dead_ahs; struct ib_event_handler event_handler; @@ -354,7 +371,7 @@ struct ipoib_dev_priv { struct ipoib_ethtool_st ethtool; struct timer_list poll_timer; - struct ipoib_lro lro; + struct ipoib_rca *rca; }; struct ipoib_ah { @@ -427,6 +444,9 @@ extern struct workqueue_struct *ipoib_workqueue; int ipoib_poll(struct napi_struct *napi, int budget); void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr); void ipoib_send_comp_handler(struct ib_cq *cq, void *dev_ptr); +void ipoib_rca_rx_handler(struct ib_cq *cq, void *ptr); +int rca_ring_poll(struct napi_struct *napi, int budget); +void ipoib_rca_qps_to_err(struct net_device *dev); struct ipoib_ah *ipoib_create_ah(struct net_device *dev, struct ib_pd *pd, struct ib_ah_attr *attr); @@ -495,6 +515,7 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, int ipoib_init_qp(struct net_device *dev); int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca); void ipoib_transport_dev_cleanup(struct net_device *dev); +int ipoib_rca_init(struct net_device *dev); void ipoib_event(struct ib_event_handler *handler, struct ib_event *record); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 0f2d304..b3223d8 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1441,6 +1441,9 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, "will cause multicast packet drops\n"); rtnl_lock(); + dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; + dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; + dev->dev_addr[3] = priv->qp->qp_num & 0xff; dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); rtnl_unlock(); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; @@ -1453,6 +1456,11 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, clear_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); rtnl_lock(); + if (priv->rca) { + dev->dev_addr[1] = (priv->rca->rca_qp->qp_num >> 16) & 0xff; + dev->dev_addr[2] = (priv->rca->rca_qp->qp_num >> 8) & 0xff; + dev->dev_addr[3] = priv->rca->rca_qp->qp_num & 0xff; + } if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; if (priv->hca_caps & IB_DEVICE_UD_TSO) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c index 66af5c1..2c64825 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c @@ -60,6 +60,8 @@ static int ipoib_set_coalesce(struct net_device *dev, { struct ipoib_dev_priv *priv = netdev_priv(dev); int ret; + int i; + struct rca_ring_ctx *ring; /* * Since IPoIB uses a single CQ for both rx and tx, we assume @@ -78,6 +80,19 @@ static int ipoib_set_coalesce(struct net_device *dev, return ret; } + if (priv->rca) { + for (i = 0; i < priv->rca->nrings; ++i) { + ring = &priv->rca->ring_arr[i]; + ret = ib_modify_cq(ring->cq, coal->rx_max_coalesced_frames, + coal->rx_coalesce_usecs); + if (ret && ret != -ENOSYS) { + ipoib_warn(priv, "failed modifying CQ for ring %d (%d)\n", i, ret); + return ret; + } + + } + } + coal->tx_coalesce_usecs = coal->rx_coalesce_usecs; coal->tx_max_coalesced_frames = coal->rx_max_coalesced_frames; priv->ethtool.coalesce_usecs = coal->rx_coalesce_usecs; @@ -110,21 +125,57 @@ static int ipoib_get_sset_count(struct net_device *dev, int sset) } } +static u64 get_ring_aggr(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + u64 result; + int i; + + result = priv->rx_ring.lro.lro_mgr.stats.aggregated; + + if (!priv->rca) + return result; + + for (i = 0; i < priv->rca->nrings; ++i) + result += priv->rca->ring_arr[i].lro.lro_mgr.stats.aggregated; + + return result; +} + +static u64 get_ring_flushed(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + u64 result; + int i; + + result = priv->rx_ring.lro.lro_mgr.stats.flushed; + + if (!priv->rca) + return result; + + for (i = 0; i < priv->rca->nrings; ++i) + result += priv->rca->ring_arr[i].lro.lro_mgr.stats.flushed; + + return result; +} + static void ipoib_get_ethtool_stats(struct net_device *dev, struct ethtool_stats *stats, uint64_t *data) { struct ipoib_dev_priv *priv = netdev_priv(dev); int index = 0; + u64 aggr = get_ring_aggr(dev); + u64 flushed = get_ring_flushed(dev); /* Get LRO statistics */ - data[index++] = priv->lro.lro_mgr.stats.aggregated; - data[index++] = priv->lro.lro_mgr.stats.flushed; - if (priv->lro.lro_mgr.stats.flushed) - data[index++] = priv->lro.lro_mgr.stats.aggregated / - priv->lro.lro_mgr.stats.flushed; + data[index++] = aggr; + data[index++] = flushed; + if (flushed) + data[index++] = aggr / flushed; else data[index++] = 0; - data[index++] = priv->lro.lro_mgr.stats.no_desc; + + data[index++] = priv->rx_ring.lro.lro_mgr.stats.no_desc; } static const struct ethtool_ops ipoib_ethtool_ops = { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 66cafa2..e8f686c 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -125,29 +125,32 @@ static void ipoib_ud_skb_put_frags(struct ipoib_dev_priv *priv, } -static int ipoib_ib_post_receive(struct net_device *dev, int id) +static int ipoib_ib_post_receive(struct net_device *dev, + struct rca_ring_ctx *ring, int id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_recv_wr *bad_wr; int ret; - priv->rx_wr.wr_id = id | IPOIB_OP_RECV; - priv->rx_sge[0].addr = priv->rx_ring[id].mapping[0]; - priv->rx_sge[1].addr = priv->rx_ring[id].mapping[1]; + ring->rx_wr.wr_id = id | IPOIB_OP_RECV; + ring->rx_sge[0].addr = ring->rx_ring[id].mapping[0]; + ring->rx_sge[1].addr = ring->rx_ring[id].mapping[1]; - ret = ib_post_recv(priv->qp, &priv->rx_wr, &bad_wr); + ret = ib_post_recv(ring->qp, &ring->rx_wr, &bad_wr); if (unlikely(ret)) { - ipoib_warn(priv, "receive failed for buf %d (%d)\n", id, ret); - ipoib_ud_dma_unmap_rx(priv, priv->rx_ring[id].mapping); - dev_kfree_skb_any(priv->rx_ring[id].skb); - priv->rx_ring[id].skb = NULL; + ipoib_warn(priv, "%s failed for buf %d (%d)\n", __func__, id, ret); + ipoib_ud_dma_unmap_rx(priv, ring->rx_ring[id].mapping); + dev_kfree_skb_any(ring->rx_ring[id].skb); + ring->rx_ring[id].skb = NULL; } return ret; } -static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) +static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, + struct ipoib_rx_buf *rx_ring, + int id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct sk_buff *skb; @@ -170,7 +173,7 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) */ skb_reserve(skb, 4); - mapping = priv->rx_ring[id].mapping; + mapping = rx_ring[id].mapping; mapping[0] = ib_dma_map_single(priv->ca, skb->data, buf_size, DMA_FROM_DEVICE); if (unlikely(ib_dma_mapping_error(priv->ca, mapping[0]))) @@ -188,7 +191,7 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id) goto partial_error; } - priv->rx_ring[id].skb = skb; + rx_ring[id].skb = skb; return skb; partial_error: @@ -204,11 +207,31 @@ static int ipoib_ib_post_receives(struct net_device *dev) int i; for (i = 0; i < ipoib_recvq_size; ++i) { - if (!ipoib_alloc_rx_skb(dev, i)) { + if (!ipoib_alloc_rx_skb(dev, priv->rx_ring.rx_ring, i)) { ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); return -ENOMEM; } - if (ipoib_ib_post_receive(dev, i)) { + if (ipoib_ib_post_receive(dev, &priv->rx_ring, i)) { + ipoib_warn(priv, "%s failed for buf %d\n", __func__, i); + return -EIO; + } + } + + return 0; +} + +static int ipoib_post_ring_receives(struct net_device *dev, + struct rca_ring_ctx *ring) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + for (i = 0; i < ipoib_recvq_size; ++i) { + if (!ipoib_alloc_rx_skb(dev, ring->rx_ring, i)) { + ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); + return -ENOMEM; + } + if (ipoib_ib_post_receive(dev, ring, i)) { ipoib_warn(priv, "ipoib_ib_post_receive failed for buf %d\n", i); return -EIO; } @@ -217,7 +240,24 @@ static int ipoib_ib_post_receives(struct net_device *dev) return 0; } -static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) +static int ipoib_rca_post_receives(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int err; + int i; + + for (i = 0; i < priv->rca->nrings; ++i) { + err = ipoib_post_ring_receives(dev, &priv->rca->ring_arr[i]); + if (err) + return err; + } + + return 0; +} + +static void ipoib_ib_handle_rx_wc(struct net_device *dev, + struct rca_ring_ctx *ring, + struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id & ~IPOIB_OP_RECV; @@ -233,16 +273,16 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) return; } - skb = priv->rx_ring[wr_id].skb; + skb = ring->rx_ring[wr_id].skb; if (unlikely(wc->status != IB_WC_SUCCESS)) { if (wc->status != IB_WC_WR_FLUSH_ERR) ipoib_warn(priv, "failed recv event " "(status=%d, wrid=%d vend_err %x)\n", wc->status, wr_id, wc->vendor_err); - ipoib_ud_dma_unmap_rx(priv, priv->rx_ring[wr_id].mapping); + ipoib_ud_dma_unmap_rx(priv, ring->rx_ring[wr_id].mapping); dev_kfree_skb_any(skb); - priv->rx_ring[wr_id].skb = NULL; + ring->rx_ring[wr_id].skb = NULL; return; } @@ -253,15 +293,15 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) if (wc->slid == priv->local_lid && wc->src_qp == priv->qp->qp_num) goto repost; - memcpy(mapping, priv->rx_ring[wr_id].mapping, + memcpy(mapping, ring->rx_ring[wr_id].mapping, IPOIB_UD_RX_SG * sizeof *mapping); /* * If we can't allocate a new RX buffer, dump * this packet and reuse the old buffer. */ - if (unlikely(!ipoib_alloc_rx_skb(dev, wr_id))) { - ++dev->stats.rx_dropped; + if (unlikely(!ipoib_alloc_rx_skb(dev, ring->rx_ring, wr_id))) { + ++ring->rx_dropped; goto repost; } @@ -278,8 +318,8 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb_pull(skb, IPOIB_ENCAP_LEN); dev->last_rx = jiffies; - ++dev->stats.rx_packets; - dev->stats.rx_bytes += skb->len; + ++ring->rx_packets; + ring->rx_bytes += skb->len; skb->dev = dev; /* XXX get correct PACKET_ type here */ @@ -289,12 +329,12 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb->ip_summed = CHECKSUM_UNNECESSARY; if (dev->features & NETIF_F_LRO) - lro_receive_skb(&priv->lro.lro_mgr, skb, NULL); + lro_receive_skb(&ring->lro.lro_mgr, skb, NULL); else netif_receive_skb(skb); repost: - if (unlikely(ipoib_ib_post_receive(dev, wr_id))) + if (unlikely(ipoib_ib_post_receive(dev, ring, wr_id))) ipoib_warn(priv, "ipoib_ib_post_receive failed " "for buf %d\n", wr_id); } @@ -408,10 +448,10 @@ static int poll_tx(struct ipoib_dev_priv *priv) return n == MAX_SEND_CQE; } -int ipoib_poll(struct napi_struct *napi, int budget) +static int rx_poll(struct napi_struct *napi, int budget, struct rca_ring_ctx *ring) { - struct ipoib_dev_priv *priv = container_of(napi, struct ipoib_dev_priv, napi); - struct net_device *dev = priv->dev; + struct net_device *dev = napi->dev; + struct ipoib_dev_priv *priv = netdev_priv(dev); int done; int t; int n, i; @@ -423,17 +463,17 @@ poll_more: int max = (budget - done); t = min(IPOIB_NUM_WC, max); - n = ib_poll_cq(priv->recv_cq, t, priv->ibwc); + n = ib_poll_cq(ring->cq, t, ring->ibwc); for (i = 0; i < n; i++) { - struct ib_wc *wc = priv->ibwc + i; + struct ib_wc *wc = ring->ibwc + i; if (wc->wr_id & IPOIB_OP_RECV) { ++done; if (wc->wr_id & IPOIB_OP_CM) ipoib_cm_handle_rx_wc(dev, wc); else - ipoib_ib_handle_rx_wc(dev, wc); + ipoib_ib_handle_rx_wc(dev, ring, wc); } else ipoib_cm_handle_tx_wc(priv->dev, wc); } @@ -444,10 +484,10 @@ poll_more: if (done < budget) { if (dev->features & NETIF_F_LRO) - lro_flush_all(&priv->lro.lro_mgr); + lro_flush_all(&ring->lro.lro_mgr); netif_rx_complete(dev, napi); - if (unlikely(ib_req_notify_cq(priv->recv_cq, + if (unlikely(ib_req_notify_cq(ring->cq, IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)) && netif_rx_reschedule(dev, napi)) @@ -457,6 +497,28 @@ poll_more: return done; } +int rca_ring_poll(struct napi_struct *napi, int budget) +{ + struct rca_ring_ctx *ring = container_of(napi, struct rca_ring_ctx, napi); + + return rx_poll(napi, budget, ring); +} + +int ipoib_poll(struct napi_struct *napi, int budget) +{ + struct ipoib_dev_priv *priv = netdev_priv(napi->dev); + + return rx_poll(napi, budget, &priv->rx_ring); +} + +void ipoib_rca_rx_handler(struct ib_cq *cq, void *ptr) +{ + struct rca_ring_ctx *ring = ptr; + struct net_device *dev = ring->napi.dev; + + netif_rx_schedule(dev, &ring->napi); +} + void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) { struct net_device *dev = dev_ptr; @@ -677,6 +739,14 @@ int ipoib_ib_dev_open(struct net_device *dev) return -1; } + if (priv->rca) { + ret = ipoib_rca_init(dev); + if (!ret) + ipoib_rca_post_receives(dev); + else + ipoib_warn(priv, "ipoib_rca_init returned %d\n", ret); + } + clear_bit(IPOIB_STOP_REAPER, &priv->flags); queue_delayed_work(ipoib_workqueue, &priv->ah_reap_task, round_jiffies_relative(HZ)); @@ -748,12 +818,22 @@ static int recvs_pending(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); int pending = 0; - int i; + int i, j; for (i = 0; i < ipoib_recvq_size; ++i) - if (priv->rx_ring[i].skb) + if (priv->rx_ring.rx_ring[i].skb) ++pending; + if (!priv->rca) + goto out; + + for (j = 0; j < priv->rca->nrings; ++j) { + for (i = 0; i < ipoib_recvq_size; ++i) + if (priv->rca->ring_arr[j].rx_ring[i].skb) + ++pending; + } + +out: return pending; } @@ -762,23 +842,23 @@ void ipoib_drain_cq(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int i, n; do { - n = ib_poll_cq(priv->recv_cq, IPOIB_NUM_WC, priv->ibwc); + n = ib_poll_cq(priv->recv_cq, IPOIB_NUM_WC, priv->rx_ring.ibwc); for (i = 0; i < n; ++i) { /* * Convert any successful completions to flush * errors to avoid passing packets up the * stack after bringing the device down. */ - if (priv->ibwc[i].status == IB_WC_SUCCESS) - priv->ibwc[i].status = IB_WC_WR_FLUSH_ERR; + if (priv->rx_ring.ibwc[i].status == IB_WC_SUCCESS) + priv->rx_ring.ibwc[i].status = IB_WC_WR_FLUSH_ERR; - if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) { - if (priv->ibwc[i].wr_id & IPOIB_OP_CM) - ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); + if (priv->rx_ring.ibwc[i].wr_id & IPOIB_OP_RECV) { + if (priv->rx_ring.ibwc[i].wr_id & IPOIB_OP_CM) + ipoib_cm_handle_rx_wc(dev, priv->rx_ring.ibwc + i); else - ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); + ipoib_ib_handle_rx_wc(dev, &priv->rx_ring, priv->rx_ring.ibwc + i); } else - ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); + ipoib_cm_handle_tx_wc(dev, priv->rx_ring.ibwc + i); } } while (n == IPOIB_NUM_WC); @@ -786,6 +866,34 @@ void ipoib_drain_cq(struct net_device *dev) ; /* nothing */ } +void ipoib_drain_rca_rx_wrs(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct rca_ring_ctx *ring; + int i, n, k; + + if (!priv->rca) + return; + + for (k = 0; k < priv->rca->nrings; ++k) { + ring = &priv->rca->ring_arr[k]; + do { + n = ib_poll_cq(ring->cq, IPOIB_NUM_WC, ring->ibwc); + for (i = 0; i < n; ++i) { + /* + * Convert any successful completions to flush + * errors to avoid passing packets up the + * stack after bringing the device down. + */ + if (ring->ibwc[i].status == IB_WC_SUCCESS) + ring->ibwc[i].status = IB_WC_WR_FLUSH_ERR; + + ipoib_ib_handle_rx_wc(dev, ring, ring->ibwc + i); + } + } while (n == IPOIB_NUM_WC); + } +} + int ipoib_ib_dev_stop(struct net_device *dev, int flush) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -806,6 +914,8 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) if (ib_modify_qp(priv->qp, &qp_attr, IB_QP_STATE)) ipoib_warn(priv, "Failed to modify QP to ERROR state\n"); + ipoib_rca_qps_to_err(dev); + /* Wait for all sends and receives to complete */ begin = jiffies; @@ -830,11 +940,11 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) for (i = 0; i < ipoib_recvq_size; ++i) { struct ipoib_rx_buf *rx_req; - rx_req = &priv->rx_ring[i]; + rx_req = &priv->rx_ring.rx_ring[i]; if (!rx_req->skb) continue; ipoib_ud_dma_unmap_rx(priv, - priv->rx_ring[i].mapping); + priv->rx_ring.rx_ring[i].mapping); dev_kfree_skb_any(rx_req->skb); rx_req->skb = NULL; } @@ -843,6 +953,8 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) } ipoib_drain_cq(dev); + ipoib_drain_rca_rx_wrs(dev); + msleep(1); } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 8be9ea0..796cf86 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -100,6 +100,31 @@ static struct ib_client ipoib_client = { .remove = ipoib_remove_one }; + +static void enable_rca_napi(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + if (!priv->rca) + return; + + for (i = 0; i < priv->rca->nrings; ++i) + napi_enable(&priv->rca->ring_arr[i].napi); +} + +static void disable_rca_napi(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + if (!priv->rca) + return; + + for (i = 0; i < priv->rca->nrings; ++i) + napi_disable(&priv->rca->ring_arr[i].napi); +} + int ipoib_open(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -107,6 +132,7 @@ int ipoib_open(struct net_device *dev) ipoib_dbg(priv, "bringing up interface\n"); napi_enable(&priv->napi); + enable_rca_napi(dev); set_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags); if (ipoib_pkey_dev_delay_open(dev)) @@ -114,12 +140,14 @@ int ipoib_open(struct net_device *dev) if (ipoib_ib_dev_open(dev)) { napi_disable(&priv->napi); + disable_rca_napi(dev); return -EINVAL; } if (ipoib_ib_dev_up(dev)) { ipoib_ib_dev_stop(dev, 1); napi_disable(&priv->napi); + enable_rca_napi(dev); return -EINVAL; } @@ -153,6 +181,7 @@ static int ipoib_stop(struct net_device *dev) clear_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags); napi_disable(&priv->napi); + disable_rca_napi(dev); netif_stop_queue(dev); @@ -921,9 +950,9 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) struct ipoib_dev_priv *priv = netdev_priv(dev); /* Allocate RX/TX "rings" to hold queued skbs */ - priv->rx_ring = kzalloc(ipoib_recvq_size * sizeof *priv->rx_ring, + priv->rx_ring.rx_ring = kzalloc(ipoib_recvq_size * sizeof *priv->rx_ring.rx_ring, GFP_KERNEL); - if (!priv->rx_ring) { + if (!priv->rx_ring.rx_ring) { printk(KERN_WARNING "%s: failed to allocate RX ring (%d entries)\n", ca->name, ipoib_recvq_size); goto out; @@ -948,7 +977,7 @@ out_tx_ring_cleanup: vfree(priv->tx_ring); out_rx_ring_cleanup: - kfree(priv->rx_ring); + kfree(priv->rx_ring.rx_ring); out: return -ENOMEM; @@ -969,10 +998,10 @@ void ipoib_dev_cleanup(struct net_device *dev) ipoib_ib_dev_cleanup(dev); - kfree(priv->rx_ring); + kfree(priv->rx_ring.rx_ring); vfree(priv->tx_ring); - priv->rx_ring = NULL; + priv->rx_ring.rx_ring = NULL; priv->tx_ring = NULL; } @@ -1017,15 +1046,33 @@ static int get_skb_hdr(struct sk_buff *skb, void **iphdr, return 0; } -static void ipoib_lro_setup(struct ipoib_dev_priv *priv) +static void ipoib_lro_setup(struct ipoib_lro *lro, struct net_device *dev) +{ + lro->lro_mgr.max_aggr = lro_max_aggr; + lro->lro_mgr.max_desc = IPOIB_MAX_LRO_DESCRIPTORS; + lro->lro_mgr.lro_arr = lro->lro_desc; + lro->lro_mgr.get_skb_header = get_skb_hdr; + lro->lro_mgr.features = LRO_F_NAPI; + lro->lro_mgr.dev = dev; + lro->lro_mgr.ip_summed_aggr = CHECKSUM_UNNECESSARY; +} + +struct net_device_stats* ipoib_get_stats(struct net_device *dev) { - priv->lro.lro_mgr.max_aggr = lro_max_aggr; - priv->lro.lro_mgr.max_desc = IPOIB_MAX_LRO_DESCRIPTORS; - priv->lro.lro_mgr.lro_arr = priv->lro.lro_desc; - priv->lro.lro_mgr.get_skb_header = get_skb_hdr; - priv->lro.lro_mgr.features = LRO_F_NAPI; - priv->lro.lro_mgr.dev = priv->dev; - priv->lro.lro_mgr.ip_summed_aggr = CHECKSUM_UNNECESSARY; + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + dev->stats.rx_packets = priv->rx_ring.rx_packets; + dev->stats.rx_dropped = priv->rx_ring.rx_dropped; + dev->stats.rx_bytes = priv->rx_ring.rx_bytes; + if (priv->rca) { + for (i = 0; i < priv->rca->nrings; ++i) { + dev->stats.rx_packets += priv->rca->ring_arr[i].rx_packets; + dev->stats.rx_dropped += priv->rca->ring_arr[i].rx_dropped; + dev->stats.rx_bytes += priv->rca->ring_arr[i].rx_bytes; + } + } + return &dev->stats; } static void ipoib_setup(struct net_device *dev) @@ -1040,6 +1087,7 @@ static void ipoib_setup(struct net_device *dev) dev->header_ops = &ipoib_header_ops; dev->set_multicast_list = ipoib_set_mcast_list; dev->neigh_setup = ipoib_neigh_setup_dev; + dev->get_stats = ipoib_get_stats; ipoib_set_ethtool_ops(dev); @@ -1067,7 +1115,7 @@ static void ipoib_setup(struct net_device *dev) priv->dev = dev; - ipoib_lro_setup(priv); + ipoib_lro_setup(&priv->rx_ring.lro, dev); spin_lock_init(&priv->lock); spin_lock_init(&priv->tx_lock); @@ -1190,6 +1238,16 @@ int ipoib_add_pkey_attr(struct net_device *dev) return device_create_file(&dev->dev, &dev_attr_pkey); } +static void ipoib_register_rings(struct ipoib_dev_priv *priv) +{ + int i; + + for (i = 0; i < priv->rca->nrings; ++i) { + netif_napi_add(priv->dev, &priv->rca->ring_arr[i].napi, rca_ring_poll, 100); + ipoib_lro_setup(&priv->rca->ring_arr[i].lro, priv->dev); + } +} + static struct net_device *ipoib_add_port(const char *format, struct ib_device *hca, u8 port) { @@ -1273,6 +1331,9 @@ static struct net_device *ipoib_add_port(const char *format, goto device_init_failed; } + if (priv->rca) + ipoib_register_rings(priv); + INIT_IB_EVENT_HANDLER(&priv->event_handler, priv->ca, ipoib_event); result = ib_register_event_handler(&priv->event_handler); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 6832511..6506e23 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -33,6 +33,35 @@ #include "ipoib.h" +static int rca; +module_param(rca, bool, 0444); +MODULE_PARM_DESC(rca, "Enable RCA (Receive Core Affinity)"); + +static int set_rca_qkey(struct ipoib_dev_priv *priv) +{ + struct ib_qp_attr *qp_attr; + int ret = -ENOMEM; + int i; + + if (!priv->rca) + return 0; + + qp_attr = kmalloc(sizeof *qp_attr, GFP_KERNEL); + if (!qp_attr) + return -ENOMEM; + + qp_attr->qkey = priv->qkey; + for (i = 0; i < priv->rca->nrings; ++i) { + ret = ib_modify_qp(priv->rca->ring_arr[i].qp, qp_attr, IB_QP_QKEY); + if (ret) + goto out; + } + +out: + kfree(qp_attr); + return ret; +} + int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -60,6 +89,12 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int ipoib_warn(priv, "failed to modify QP, ret = %d\n", ret); goto out; } + + ret = set_rca_qkey(priv); + if (ret) { + ipoib_warn(priv, "failed to modify RCA QPs qkey, ret = %d\n", ret); + goto out; + } } /* attach QP to multicast group */ @@ -126,6 +161,308 @@ out_fail: return ret; } +static void qp2reset(struct ipoib_dev_priv *priv, struct ib_qp *qp) +{ + struct ib_qp_attr qp_attr; + + qp_attr.qp_state = IB_QPS_RESET; + if (ib_modify_qp(qp, &qp_attr, IB_QP_STATE)) + ipoib_warn(priv, "Failed to modify QP to RESET state\n"); +} + +static void qp2err(struct ipoib_dev_priv *priv, struct ib_qp *qp) +{ + struct ib_qp_attr qp_attr; + + qp_attr.qp_state = IB_QPS_ERR; + if (ib_modify_qp(qp, &qp_attr, IB_QP_STATE)) + ipoib_warn(priv, "Failed to modify QP to ERROR state\n"); +} + +static int modify_to_rts(struct ipoib_dev_priv *priv, struct ib_qp *qp) // ?? need to modify to rtr +{ + int ret; + struct ib_qp_attr qp_attr; + int attr_mask; + + qp_attr.qp_state = IB_QPS_INIT; + qp_attr.qkey = 0; + qp_attr.port_num = priv->port; + qp_attr.pkey_index = priv->pkey_index; + attr_mask = + IB_QP_QKEY | + IB_QP_PORT | + IB_QP_PKEY_INDEX | + IB_QP_STATE; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to init, ret = %d\n", ret); + goto out_fail; + } + + qp_attr.qp_state = IB_QPS_RTR; + /* Can't set this in a INIT->RTR transition */ + attr_mask &= ~IB_QP_PORT; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to RTR, ret = %d\n", ret); + goto out_fail; + } + + qp_attr.qp_state = IB_QPS_RTS; + qp_attr.sq_psn = 0; + attr_mask |= IB_QP_SQ_PSN; + attr_mask &= ~IB_QP_PKEY_INDEX; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to RTS, ret = %d\n", ret); + goto out_fail; + } + + return 0; + +out_fail: + qp2reset(priv, qp); + return ret; +} + +static int prepare_rca_qp(struct ipoib_dev_priv *priv, struct ib_qp *qp) +{ + int ret; + struct ib_qp_attr qp_attr; + int attr_mask; + + qp_attr.qp_state = IB_QPS_INIT; + qp_attr.qkey = 0; + qp_attr.port_num = priv->port; + qp_attr.pkey_index = priv->pkey_index; + qp_attr.rca.base_qpn = priv->rca->ring_arr[0].qp->qp_num; + qp_attr.rca.num_qpn = priv->rca->nrings; + qp_attr.rca.default_qpn = priv->qp->qp_num; + + attr_mask = + IB_QP_QKEY | + IB_QP_PORT | + IB_QP_PKEY_INDEX | + IB_QP_STATE | + IB_QP_RCA; + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to init, ret = %d\n", ret); + goto out_fail; + } + + qp_attr.qp_state = IB_QPS_RTR; + /* Can't set this in a INIT->RTR transition */ + attr_mask &= ~(IB_QP_PORT | IB_QP_RCA); + ret = ib_modify_qp(qp, &qp_attr, attr_mask); + if (ret) { + ipoib_warn(priv, "failed to modify QP to RTR, ret = %d\n", ret); + goto out_fail; + } + + return 0; + +out_fail: + qp2reset(priv, qp); + return ret; +} + +int ipoib_rca_init(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int err; + int i; + + for (i = 0; i < priv->rca->nrings; ++i) { + err = modify_to_rts(priv, priv->rca->ring_arr[i].qp); + if (err) { + ipoib_warn(priv, "ring[%d] failed modify to RTS\n", i); + goto exit_err; + } + } + + err = prepare_rca_qp(priv, priv->rca->rca_qp); + if (err) + goto exit_err; + + return 0; + +exit_err: + for (--i; i >= 0; --i) + qp2reset(priv, priv->rca->ring_arr[i].qp); + + return err; +} + +void ipoib_rca_qps_to_err(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int i; + + if (!priv->rca) + return; + + for (i = 0; i < priv->rca->nrings; ++i) + qp2err(priv, priv->rca->ring_arr[i].qp); +} + +static void destroy_rca_resources(struct ipoib_dev_priv *priv) +{ + int i; + int err; + + for (i = 0; i < priv->rca->nrings; ++i) { + if (priv->rca->ring_arr[i].qp) { + err = ib_destroy_qp(priv->rca->ring_arr[i].qp); + if (err) + printk(KERN_WARNING "%s: failed to destroy qp[%d], err = %d\n", + __func__, i, err); + } + + if (priv->rca->ring_arr[i].cq) { + err = ib_destroy_cq(priv->rca->ring_arr[i].cq); + if (err) + printk(KERN_WARNING "%s: failed to destroy cq[%d], err = %d\n", + __func__, i, err); + } + + kfree(priv->rca->ring_arr[i].rx_ring); + } + + if (priv->rca->rca_qp) + if (ib_destroy_qp(priv->rca->rca_qp)) + printk(KERN_WARNING "failed to destroy rca qp\n"); + + kfree(priv->rca); + priv->rca = NULL; +} + +static void qpi_rx_event_handler(struct ib_event *event, void *ctx) +{ + struct ipoib_dev_priv *priv = ctx; + + ipoib_warn(priv, "got event %d on RCA QP. QPN = 0x%x\n", event->event, + event->element.qp->qp_num); +} + +static void rca_qp_rx_event_handler(struct ib_event *event, void *ctx) +{ + struct ipoib_dev_priv *priv = ctx; + + ipoib_warn(priv, "RCA QP got event %d. QPN = 0x%x\n", event->event, + event->element.qp->qp_num); +} + +static void init_ring(struct ipoib_dev_priv *priv, struct rca_ring_ctx *ring) +{ + ring->rx_sge[0].lkey = priv->mr->lkey; + if (ipoib_ud_need_sg(priv->max_ib_mtu)) { + ring->rx_sge[0].length = IPOIB_UD_HEAD_SIZE; + ring->rx_sge[1].length = PAGE_SIZE; + ring->rx_sge[1].lkey = priv->mr->lkey; + ring->rx_wr.num_sge = IPOIB_UD_RX_SG; + } else { + ring->rx_sge[0].length = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); + ring->rx_wr.num_sge = 1; + } + ring->rx_wr.next = NULL; + ring->rx_wr.sg_list = ring->rx_sge; +} + +static int create_rca_resources(struct ipoib_dev_priv *priv, struct ib_device *ca) +{ + struct ib_qp_init_attr *attr; + int err; + int i; + int num_comp_vectors = ca->num_comp_vectors; + struct ib_qp **qp_arr; + + priv->rca = kzalloc(sizeof *priv->rca + num_comp_vectors * + sizeof *priv->rca->ring_arr, GFP_KERNEL); + if (!priv->rca) + return -ENOMEM; + + attr = kzalloc(num_comp_vectors * sizeof *attr, GFP_KERNEL); + if (!attr) { + err = -ENOMEM; + goto exit_err; + } + qp_arr = kmalloc(num_comp_vectors * sizeof *qp_arr, GFP_KERNEL); + if (!qp_arr) { + kfree(attr); + err = -ENOMEM; + goto exit_err; + } + + for (i = 0; i < num_comp_vectors; ++i) { + priv->rca->ring_arr[i].rx_ring = kzalloc(sizeof(struct ipoib_rx_buf) * + ipoib_recvq_size, GFP_KERNEL); + if (!priv->rca->ring_arr[i].rx_ring) { + err = -ENOMEM; + goto exit_free; + } + + init_ring(priv, &priv->rca->ring_arr[i]); + + priv->rca->ring_arr[i].cq = ib_create_cq(ca, ipoib_rca_rx_handler, NULL, + &priv->rca->ring_arr[i], ipoib_recvq_size, i); + if (IS_ERR(priv->rca->ring_arr[i].cq)) { + err = PTR_ERR(priv->rca->ring_arr[i].cq); + priv->rca->ring_arr[i].cq = NULL; + goto exit_free; + } + + err = ib_req_notify_cq(priv->rca->ring_arr[i].cq, IB_CQ_NEXT_COMP); + if (err) + goto exit_free; + + attr[i].event_handler = qpi_rx_event_handler; + attr[i].qp_context = priv; + attr[i].sq_sig_type = IB_SIGNAL_ALL_WR; + attr[i].qp_type = IB_QPT_UD; + attr[i].cap.max_recv_wr = ipoib_recvq_size; + attr[i].cap.max_recv_sge = IPOIB_UD_RX_SG; + attr[i].recv_cq = attr[i].send_cq = priv->rca->ring_arr[i].cq; + } + + err = ib_create_qp_range(priv->pd, attr, num_comp_vectors, + 1 << ilog2(num_comp_vectors), qp_arr); + if (err) + goto exit_free; + + for (i = 0; i < num_comp_vectors; ++i) + priv->rca->ring_arr[i].qp = qp_arr[i]; + + memset(attr, 0, sizeof *attr); + attr[0].event_handler = rca_qp_rx_event_handler; + attr[0].qp_context = priv; + attr[0].qp_type = IB_QPT_UD; + attr[0].recv_cq = attr[0].send_cq = priv->recv_cq; + attr[0].create_flags = IB_QP_CREATE_IPOIB_RCA; + priv->rca->rca_qp = ib_create_qp(priv->pd, attr); + if (IS_ERR(priv->rca->rca_qp)) { + err = PTR_ERR(priv->rca->rca_qp); + priv->rca->rca_qp = NULL; + goto exit_free; + } + + kfree(qp_arr); + kfree(attr); + priv->rca->nrings = num_comp_vectors; + + return 0; + +exit_free: + kfree(qp_arr); + kfree(attr); + +exit_err: + destroy_rca_resources(priv); + + return err; +} + int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -139,7 +476,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_UD }; - + u32 hw_qpn; int ret, size; int i; @@ -199,10 +536,6 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_send_cq; } - priv->dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; - priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; - priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) priv->tx_sge[i].lkey = priv->mr->lkey; @@ -210,18 +543,32 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; - priv->rx_sge[0].lkey = priv->mr->lkey; + priv->rx_ring.rx_sge[0].lkey = priv->mr->lkey; if (ipoib_ud_need_sg(priv->max_ib_mtu)) { - priv->rx_sge[0].length = IPOIB_UD_HEAD_SIZE; - priv->rx_sge[1].length = PAGE_SIZE; - priv->rx_sge[1].lkey = priv->mr->lkey; - priv->rx_wr.num_sge = IPOIB_UD_RX_SG; + priv->rx_ring.rx_sge[0].length = IPOIB_UD_HEAD_SIZE; + priv->rx_ring.rx_sge[1].length = PAGE_SIZE; + priv->rx_ring.rx_sge[1].lkey = priv->mr->lkey; + priv->rx_ring.rx_wr.num_sge = IPOIB_UD_RX_SG; } else { - priv->rx_sge[0].length = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); - priv->rx_wr.num_sge = 1; + priv->rx_ring.rx_sge[0].length = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu); + priv->rx_ring.rx_wr.num_sge = 1; + } + priv->rx_ring.rx_wr.next = NULL; + priv->rx_ring.rx_wr.sg_list = priv->rx_ring.rx_sge; + priv->rx_ring.qp = priv->qp; + priv->rx_ring.cq = priv->recv_cq; + + hw_qpn = priv->qp->qp_num; + if (rca && priv->hca_caps & IB_DEVICE_IPOIB_RCA) { + if (create_rca_resources(priv, ca)) + printk(KERN_WARNING "not using RCA for device %s\n", ca->name); + else + hw_qpn = priv->rca->rca_qp->qp_num; } - priv->rx_wr.next = NULL; - priv->rx_wr.sg_list = priv->rx_sge; + + priv->dev->dev_addr[1] = (hw_qpn >> 16) & 0xff; + priv->dev->dev_addr[2] = (hw_qpn >> 8) & 0xff; + priv->dev->dev_addr[3] = hw_qpn & 0xff; return 0; @@ -244,6 +591,9 @@ void ipoib_transport_dev_cleanup(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); + if (priv->rca) + destroy_rca_resources(priv); + if (priv->qp) { if (ib_destroy_qp(priv->qp)) ipoib_warn(priv, "ib_qp_destroy failed\n"); -- 1.5.6 From jackm at dev.mellanox.co.il Thu Jul 17 08:02:57 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 17 Jul 2008 18:02:57 +0300 Subject: [ofa-general] Re: [PATCH 11/12 V2] mlx4: XRC receive-only QPs In-Reply-To: References: <200807101852.06851.jackm@dev.mellanox.co.il> Message-ID: <200807171802.57647.jackm@dev.mellanox.co.il> On Wednesday 16 July 2008 22:12, Roland Dreier wrote > I have one very specific question: > > > --- a/drivers/infiniband/hw/mlx4/cq.c > > +++ b/drivers/infiniband/hw/mlx4/cq.c > > @@ -174,7 +174,7 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector > > if (entries < 1 || entries > dev->dev->caps.max_cqes) > > return ERR_PTR(-EINVAL); > > > > - cq = kmalloc(sizeof *cq, GFP_KERNEL); > > + cq = kzalloc(sizeof *cq, GFP_KERNEL); > > if (!cq) > > return ERR_PTR(-ENOMEM); > > why is this change made in this patch? Because of the call below to mlx4_ib_create_qp -- I wanted to zero-out the entire structure to not worry about the ib (core) layer info. > > And a more general question: why do these kernel receive-only QPs affect > device-dependent low-level driver code at all. This whole patch seems > to be just doing bookkeeping that seems like it could be done in generic > code, and it leads to layering problems like: > > + cq = mlx4_ib_create_cq(ibdev, 1, 0, NULL, NULL); > + if (IS_ERR(cq)) { > + err = PTR_ERR(cq); > + goto err_cq; > + } > + cq->device = ibdev; > + cq->comp_handler = mlx4_dummy_comp_handler; > > ie mlx4 ends up duplicating code from ib_create_cq() for no reason I can > understand. In the draft addition to the IB Spec, XRC Target QPs (XRC-RCV qp's here) do not require CQ's, so we should not require such in the core. Since in ConnectX we do require every QP to have send and receive CQs, I created, for each XRC domain, a single CQ for use with XRC RCV qps in that domain, in the low level driver. At the time, I thought it best to keep these CQs completely internal to the low-level driver layer. I will think about having the low-level driver invoke ib_create_cq() instead of mlx4_ib_create_cq() -- and ditto for PDs. > I'm probably missing something but why can't all or at least most of > this be done in the generic IB code? I don't see anything related to > device HW interface anywhere in this patch. The CQ usage in XRC RCV cq's is HW specific (it is not required by the IB Spec). - Jack From tomquinn at sympatico.ca Thu Jul 17 09:22:07 2008 From: tomquinn at sympatico.ca (Mrs.Rose Wood) Date: Thu, 17 Jul 2008 12:22:07 -0400 Subject: [ofa-general] Ref: 6525/08 Message-ID: <20080717162226.QRWX1673.tomts47-srv.bellnexxia.net@toip41-bus.srvr.bell.ca> You won the sum of £450,000.00GBP from our monthly PROGRAM,held on the 1st July 2008,and released today 17th JuLY 2008.You are adviced to Contact Mr.Pinkett to claim your prize. Provide your Names,Address,Age,Occupation,Tel and Country. Email:onlineinfounit at btinternet.com Sincerely, Mrs.Rose Wood. From ralph.campbell at qlogic.com Thu Jul 17 09:59:09 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 17 Jul 2008 09:59:09 -0700 Subject: [ofa-general] [PATCH] IB/core - ib_sa_remove_one() can panic due to NULL pointer dereference In-Reply-To: References: <20080716223312.21857.21231.stgit@eng-46.mv.qlogic.com> Message-ID: <1216313949.28040.390.camel@chromite.mv.qlogic.com> Yes, I have seen it when update_sm_ah() calls ib_create_ah() which fails and leaves sm_ah == NULL. Granted that ib_create_ah() rarely fails though. On Wed, 2008-07-16 at 17:46 -0700, Roland Dreier wrote: > > If an HCA does not get a SM LID, ib_sa_remove_one() can cause a NULL > > pointer bug when it calls kref_put(). This is easy to see since > > ib_sa_add_one() sets sm_ah to NULL. > > Have you seen this in practice? Because ib_sa_add_one() calls > update_sm_ah() before returning, so ib_sa_remove_one() shouldn't see a > NULL sm_ah because of that. And ib_sa_event() schedules the update task > after setting sm_ah to NULL, and ib_sa_remove_one() does > flush_scheduled_work() so update_sm_ah() should be guaranteed to run in > that case too. > > I do see the possibility of hitting one of the failures in > update_sm_ah() and ending up not setting sm_ah, so your patch is > correct, but I'm just wondering how you hit this in practice. > > I also see what I guess is techically a bug in ib_sa_add_one(), namely > that it does > > ib_set_client_data(device, &sa_client, sa_dev); > > before > > INIT_IB_EVENT_HANDLER(&sa_dev->event_handler, device, ib_sa_event); > if (ib_register_event_handler(&sa_dev->event_handler)) > goto err; > > and if that function returns an error, then sa_dev gets freed and > ib_sa_remove_one() ends up doing a use-after-free. However with the > current code, ib_register_event_handler() can never return an error. > > I wonder if the best fix is just to make ib_register_event_handler() > a void function. > > - R. From michael.heinz at qlogic.com Thu Jul 17 10:13:03 2008 From: michael.heinz at qlogic.com (Mike Heinz) Date: Thu, 17 Jul 2008 12:13:03 -0500 Subject: [ofa-general] Proposed patches to MVAPICH and MVAPICH2 rpm spec files Message-ID: I'm not sure who the best person is to receive these changes: We've been encountering complications whe converting users to OFED 1.3 because the scripts provided for configuring the shell (mpivars.sh and mpivars.csh) don't update the library path. This can lead to MPI programs failing to link or failing to run. The fix is to modify the spec files for the RPMs for these packages so that they set the LD_LIBRARY_PATH as well as the PATH. The fix for MVAPICH-1.0.1 is this: --- mvapich.spec.orig 2008-07-16 17:06:44.000000000 -0400 +++ mvapich.spec 2008-07-16 16:49:27.000000000 -0400 @@ -300,17 +300,25 @@ if ! echo \${PATH} | grep -q %{_prefix}/bin ; then export PATH=%{_prefix}/bin:\${PATH} fi +if ! echo \${LD_LIBRARY_PATH} | grep -q %{_prefix}/lib ; then + export LD_LIBRARY_PATH=%{_prefix}/lib:%{_prefix}/lib:/shared:\${LD_LIBRARY_PATH } +fi EOF # Script for csh cat < %{build_root}/%{_prefix}/bin/%{shell_scripts_basename}.csh -if (?$path) then - if ( "\${path}" !~ *%{_prefix}/bin* ) then - setenv path %{_prefix}/bin:\$path +if ("\$path" !~ *%{_prefix}/bin) then + set path=(%{_prefix}/bin \$path) +endif + +if ("1" == "\$?LD_LIBRARY_PATH") then + if ("\$LD_LIBRARY_PATH" !~ *%{_prefix}/lib) then + setenv LD_LIBRARY_PATH %{_prefix}/lib:%{_prefix}/lib/shared:\${LD_LIBRARY_PATH} endif else - setenv path %{_prefix}/bin: + setenv LD_LIBRARY_PATH %{_prefix}/lib:%{_prefix}/lib/shared endif + EOF and the fix for MVAPICH2-1.0.3 is this: --- ../mvapich2.spec.orig 2008-07-16 17:17:10.000000000 -0400 +++ mvapich2.spec 2008-07-17 09:03:19.000000000 -0400 @@ -261,12 +261,16 @@ # Additionally, create the mpivars.[c]sh files. cat >bin/mpivars.csh < From panda at cse.ohio-state.edu Thu Jul 17 11:09:32 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Thu, 17 Jul 2008 14:09:32 -0400 (EDT) Subject: [ofa-general] Proposed patches to MVAPICH and MVAPICH2 rpm spec files In-Reply-To: Message-ID: Hi Mike, Thanks for posting these patches. In future, please feel free to post patches related to MVAPICH and MVAPICH2 to mvapich-discuss list (cc'ed in this e-mail). Pasha (cc'ed here) will take care of the changes to MVAPICH rpm spec file. Jonathan (cc'ed here) will take care of the changes to MVAPICH2 rpm spec file. Thanks, DK On Thu, 17 Jul 2008, Mike Heinz wrote: > I'm not sure who the best person is to receive these changes: We've been > encountering complications whe converting users to OFED 1.3 because the > scripts provided for configuring the shell (mpivars.sh and mpivars.csh) > don't update the library path. This can lead to MPI programs failing to > link or failing to run. The fix is to modify the spec files for the RPMs > for these packages so that they set the LD_LIBRARY_PATH as well as the > PATH. > > The fix for MVAPICH-1.0.1 is this: > > --- mvapich.spec.orig 2008-07-16 17:06:44.000000000 -0400 > +++ mvapich.spec 2008-07-16 16:49:27.000000000 -0400 > @@ -300,17 +300,25 @@ > if ! echo \${PATH} | grep -q %{_prefix}/bin ; then > export PATH=%{_prefix}/bin:\${PATH} > fi > +if ! echo \${LD_LIBRARY_PATH} | grep -q %{_prefix}/lib ; then > + export > LD_LIBRARY_PATH=%{_prefix}/lib:%{_prefix}/lib:/shared:\${LD_LIBRARY_PATH > } > +fi > EOF > > # Script for csh > cat < %{build_root}/%{_prefix}/bin/%{shell_scripts_basename}.csh > -if (?$path) then > - if ( "\${path}" !~ *%{_prefix}/bin* ) then > - setenv path %{_prefix}/bin:\$path > +if ("\$path" !~ *%{_prefix}/bin) then > + set path=(%{_prefix}/bin \$path) > +endif > + > +if ("1" == "\$?LD_LIBRARY_PATH") then > + if ("\$LD_LIBRARY_PATH" !~ *%{_prefix}/lib) then > + setenv LD_LIBRARY_PATH > %{_prefix}/lib:%{_prefix}/lib/shared:\${LD_LIBRARY_PATH} > endif > else > - setenv path %{_prefix}/bin: > + setenv LD_LIBRARY_PATH %{_prefix}/lib:%{_prefix}/lib/shared > endif > + > EOF > > > and the fix for MVAPICH2-1.0.3 is this: > > > --- ../mvapich2.spec.orig 2008-07-16 17:17:10.000000000 -0400 > +++ mvapich2.spec 2008-07-17 09:03:19.000000000 -0400 > @@ -261,12 +261,16 @@ > > # Additionally, create the mpivars.[c]sh files. > cat >bin/mpivars.csh < -if (\$?path) then > - if ( "\${path}" !~ *%{_prefix}/bin* ) then > +if ("\$path" !~ *%{_prefix}/bin) then > set path = ( %{_prefix}/bin \$path ) > endif > + > +if ("1" == "\$?LD_LIBRARY_PATH") then > + if ("\$LD_LIBRARY_PATH" !~ *%{_prefix}/lib) then > + setenv LD_LIBRARY_PATH %{_prefix}/lib:\${LD_LIBRARY_PATH} > + endif > else > - set path = ( %{_prefix}/bin ) > + setenv LD_LIBRARY_PATH %{_prefix}/lib:%{_prefix}/lib/shared > endif > > if (\$?MANPATH) then > @@ -282,7 +286,9 @@ > if ! echo \${PATH} | grep -q %{_prefix}/bin ; then > PATH=%{_prefix}/bin:\${PATH} > fi > - > +if ! echo \${LD_LIBRARY_PATH} | grep -q %{_prefix}/lib ; then > + export LD_LIBRARY_PATH=%{_prefix}/lib:\${LD_LIBRARY_PATH} > +fi > if ! echo \${MANPATH} | grep -q %{_prefix}/man ; then > MANPATH=%{_prefix}/man:\${MANPATH} > fi > > > -- > Michael Heinz > Principal Engineer, Qlogic Corporation > King of Prussia, Pennsylvania > > From nab at linux-iscsi.org Thu Jul 17 10:44:57 2008 From: nab at linux-iscsi.org (Nicholas A. Bellinger) Date: Thu, 17 Jul 2008 10:44:57 -0700 Subject: [ofa-general] [ANNOUNCE] - VHACS-VM x86_64 Alpha Preview - FOLLOWUP In-Reply-To: <1216058533.21408.83.camel@haakon2.linux-iscsi.org> References: <1216058533.21408.83.camel@haakon2.linux-iscsi.org> Message-ID: <1216316697.7988.78.camel@haakon2.linux-iscsi.org> Greetings all, Just a quick couple of updates since the VHACS-VM announcement (which happened to be a day after after v2.6.26. ;): I) The two VHACS-VM x86_64 images, have been split into 2 compressed ~300 MB images (for the running demo), and a 800 MB BUILD .vmdk disk image containing the source for VHACS v0.8.15. The three archives can be found at the same location: http://linux-iscsi.org/builds/VHACS-VM/x86_64/vmware6/ II) lio-core-2.6.git has been updated to v2.6.26 and loaded successfully on 32-bit x86 VM. Only a very minor change to get things compiled on v2.6.26. Please see http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=summary for more information. III) VHACS-VM has now successfully been able to handle node failures and restart failed cluster resource allocations (RA) on the active node, across multiple platforms using x86_64 hardware virtualization. :-) Lots of things going on this week. More to come soon. Please keep checking the http://Linux-iSCSI.org site for the latest information. Many thanks for your most valuable of time, --nab On Mon, 2008-07-14 at 11:02 -0700, Nicholas A. Bellinger wrote: > Greetings all, > > I am pleased to announce the Alpha availibility of VHACS-VM Demo for > Hardware Virtualized x86_64 running (initially) on VMWare Workstation 6. > The VHACS cluster stack also obviously runs on 'bare-metal' hardware, > but today VHACS-VM is the easiest method to get your own open storage > cloud up and running. > > What is VHACS..? > > VHACS is a Cloud Storage implementation running on Linux v2.6. VHACS is > an acrynom for Virtualization, High Availibility, and Cluster Storage. > VHACS is a combination of at least eight (8) long term OSS/Linux based > projects, along with a CLI management interface for controlling VHACS > nodes, clouds, and vservers within the VHACS cluster. Here is a quick > rundown of the projects (so far) that make up the VHACS cluster stack: > > CLUSTER: > > * ) Pacemaker The scalable High-Availability cluster resource > manager formerly part of Heartbeat > * ) OpenAIS The OpenAIS Standards Based Cluster Framework is an > OSI Certified implementation of the Service Availability Forum > Application Interface Specification (AIS) > > SERVER: > > * ) LIO-Target w/ IBLOCK LIO-Core subsystem plugin for iSCSI > Target mode export > * ) DRBD Distributed Replicated Block device for Network RAID 1 > Mirrors > * ) Linux Logical Volume Manager (LVM) ('vhacs' volume group for > creation of VHACS clouds) > > CLIENT: > > * ) Open/iSCSI iSCSI Initiator client > * ) Ext3 filesystem > * ) Linux-Vserver Linux Virtualization Layer > > I am happy to report that VHACS-VM Alpha Demo x86_64 is running 0.8.15 > VHACS code, along with the many other pieces of open source > infrastructure that the VHACS cloud builds upon, and providing > Active/Active H/A with Synchronous Data Replication iSCSI Target ERL=2 / > MC/2 export that makes up the SERVER side of the VHACS cloud. The > CLIENT side of the VHACS cloud is also running an iSCSI Initiator ext3 > mounted filesystem with Open/iSCSI on top of Intel 45 nanometer > 'Wolfdale' microprocessor silicon and DDR2 800 memory. > > Futhermore, using commerically available x86 virtualization technology > on open/closed platforms, I am very happy to report that VHACS-VM 2x > node clusters are now up and running across both OPEN (Linux x86_64) and > CLOSED (win32) x86 platforms on top of the 45nm chips with the VMMU/VMX > parts enabled. The first OS independent storage clouds running on top > of v2.6.25.9-kdb are now a reality. > > Here is a link to the code, screenshots, and all the goodies: > > http://linux-iscsi.org/index.php/VHACS-VM > http://linux-iscsi.org/index.php/VHACS > > The VHACS-VM x86_64 system images, which contain a Debian Etch system > iamges running the very latest compontents that make up the VHACS > cluster stack. Also included in the vhac64-west and vhacs64-east system > images is a complete development environment, including a KDB enabled > kernel that will allow the developer to pause the running image at any > time, examine memory, or setup a breakpoint to track down a problem. > The x86_64 system images currently require hardware x86 virtualization, > please see the wiki for more information about hardware requirements. > They are available from: > > http://linux-iscsi.org/builds/VHACS-VM/x86_64/vmware6/ > > Also, there is great interest to see VHACS-VM running on KVM and QEMU > Virtualization platforms. This is something that should not be too > difficult as long as the virtualization platform supports execution of > x86_64 system images. i386 system images are also in the works to bring > the storage cloud to legacy environments without hardware x86_64 > virtualization. > > Enjoy, > > --nab > From 05sbrooks at detroit.lib.mi.us Thu Jul 17 11:33:15 2008 From: 05sbrooks at detroit.lib.mi.us (bennie monika) Date: Thu, 17 Jul 2008 18:33:15 +0000 Subject: [ofa-general] to general Message-ID: <000701c8e84a$02e5f05d$04c82eb9@wueiygt> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.rajwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From rdreier at cisco.com Thu Jul 17 13:48:02 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 17 Jul 2008 13:48:02 -0700 Subject: [ofa-general] [PATCH] IB/core - ib_sa_remove_one() can panic due to NULL pointer dereference In-Reply-To: <20080716223312.21857.21231.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Wed, 16 Jul 2008 15:33:13 -0700") References: <20080716223312.21857.21231.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied. From vhssleeper at commerceinvest.com Thu Jul 17 19:07:50 2008 From: vhssleeper at commerceinvest.com (Marty Hulcy) Date: Thu, 17 Jul 2008 21:07:50 -0500 Subject: [ofa-general] Strength men stuff Message-ID: <001201c8e851$2be63000$00179bd4@servidor> All the health products are made especially for you The real quality for little money here -------------- next part -------------- An HTML attachment was scrubbed... URL: From yiaulehwurygw at gaza.net Thu Jul 17 21:35:11 2008 From: yiaulehwurygw at gaza.net (shirley) Date: Thu, 17 Jul 2008 20:35:11 -0800 (EDT) Subject: [ofa-general] hi from shirley Message-ID: <87408407.8614512533404.JavaMail.vmail@service1.colo.roosfan.com> Hi, i am here sitting in the internet caffe. Found your email and decided to write. I am 25 y.o.girl. I have a picture if you want. No need to reply here as this is not may email. Write me at ashirley700 at gblnation.info From 71patricia.baroche at wanadoo.fr Fri Jul 18 00:04:04 2008 From: 71patricia.baroche at wanadoo.fr (kip shuang) Date: Fri, 18 Jul 2008 07:04:04 +0000 Subject: [ofa-general] debt consolidation secured loan Message-ID: <000801c8e8b3$04941c30$b460f08a@gmddhole> Do Not consolidate your debt EliminateE it! Legally ELIMINATE your creditt card and other unsecured debt * WITHOUT ever making another payment to your creditors * WITHOUT it affecting your credit long-term * WITHOUT confrontation Visit www.drymisted[DOT]com (copy this link and replace "[DOT]" to ".") This IS NOT: * Bankruptcy * Consolidation * Or refinancing of any kind Visit here to learn how www.drymisted[DOT]com (copy this link and replace "[DOT]" to ".") * Must have a minimum of $10K in combined household unsecured debt to apply. * Must be a US resident. From vlad at lists.openfabrics.org Fri Jul 18 02:50:23 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Fri, 18 Jul 2008 02:50:23 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080718-0200 daily build status Message-ID: <20080718095023.4726DE60398@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Failed: Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080718-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From dotanba at gmail.com Fri Jul 18 01:17:36 2008 From: dotanba at gmail.com (Dotan Barak) Date: Fri, 18 Jul 2008 11:17:36 +0300 Subject: [ofa-general] ***SPAM*** [PATCH] core: Fixed several coding style issues according to checkpatch.pl Message-ID: <200807181117.37332.dotanba@gmail.com> Fixed several coding style issues according to checkpatch.pl: * removed braces {} for single statement blocks * added missing space * removed extra spaces * sizeof (u64) was changed to sizeof u64 to be consistent * assignment in a condition was resolved * opening '{' was set in the right place in a switch statement Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c index 09a2bec..0a81fac 100644 --- a/drivers/infiniband/core/addr.c +++ b/drivers/infiniband/core/addr.c @@ -364,9 +364,8 @@ static int netevent_callback(struct notifier_block *self, unsigned long event, if (event == NETEVENT_NEIGH_UPDATE) { struct neighbour *neigh = ctx; - if (neigh->nud_state & NUD_VALID) { + if (neigh->nud_state & NUD_VALID) set_timeout(jiffies); - } } return 0; } diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index 55738ee..6f14f9c 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -309,8 +309,8 @@ static void cm_free_msg(struct ib_mad_send_buf *msg) ib_free_send_mad(msg); } -static void * cm_copy_private_data(const void *private_data, - u8 private_data_len) +static void *cm_copy_private_data(const void *private_data, + u8 private_data_len) { void *data; @@ -389,7 +389,7 @@ static int cm_alloc_id(struct cm_id_private *cm_id_priv) if (!ret) next_id = ((unsigned) id + 1) & MAX_ID_MASK; spin_unlock_irqrestore(&cm.lock, flags); - } while( (ret == -EAGAIN) && idr_pre_get(&cm.local_id_table, GFP_KERNEL) ); + } while ((ret == -EAGAIN) && idr_pre_get(&cm.local_id_table, GFP_KERNEL)); cm_id_priv->id.local_id = (__force __be32)id ^ cm.random_id_operand; return ret; @@ -403,7 +403,7 @@ static void cm_free_id(__be32 local_id) spin_unlock_irq(&cm.lock); } -static struct cm_id_private * cm_get_id(__be32 local_id, __be32 remote_id) +static struct cm_id_private *cm_get_id(__be32 local_id, __be32 remote_id) { struct cm_id_private *cm_id_priv; @@ -419,7 +419,7 @@ static struct cm_id_private * cm_get_id(__be32 local_id, __be32 remote_id) return cm_id_priv; } -static struct cm_id_private * cm_acquire_id(__be32 local_id, __be32 remote_id) +static struct cm_id_private *cm_acquire_id(__be32 local_id, __be32 remote_id) { struct cm_id_private *cm_id_priv; @@ -490,7 +490,7 @@ static int be64_gt(__be64 a, __be64 b) return (__force u64) a > (__force u64) b; } -static struct cm_id_private * cm_insert_listen(struct cm_id_private *cm_id_priv) +static struct cm_id_private *cm_insert_listen(struct cm_id_private *cm_id_priv) { struct rb_node **link = &cm.listen_service_table.rb_node; struct rb_node *parent = NULL; @@ -529,9 +529,9 @@ static struct cm_id_private * cm_insert_listen(struct cm_id_private *cm_id_priv) return NULL; } -static struct cm_id_private * cm_find_listen(struct ib_device *device, - __be64 service_id, - u8 *private_data) +static struct cm_id_private *cm_find_listen(struct ib_device *device, + __be64 service_id, + u8 *private_data) { struct rb_node *node = cm.listen_service_table.rb_node; struct cm_id_private *cm_id_priv; @@ -562,8 +562,8 @@ static struct cm_id_private * cm_find_listen(struct ib_device *device, return NULL; } -static struct cm_timewait_info * cm_insert_remote_id(struct cm_timewait_info - *timewait_info) +static struct cm_timewait_info *cm_insert_remote_id(struct cm_timewait_info + *timewait_info) { struct rb_node **link = &cm.remote_id_table.rb_node; struct rb_node *parent = NULL; @@ -592,8 +592,8 @@ static struct cm_timewait_info * cm_insert_remote_id(struct cm_timewait_info return NULL; } -static struct cm_timewait_info * cm_find_remote_id(__be64 remote_ca_guid, - __be32 remote_id) +static struct cm_timewait_info *cm_find_remote_id(__be64 remote_ca_guid, + __be32 remote_id) { struct rb_node *node = cm.remote_id_table.rb_node; struct cm_timewait_info *timewait_info; @@ -615,8 +615,8 @@ static struct cm_timewait_info * cm_find_remote_id(__be64 remote_ca_guid, return NULL; } -static struct cm_timewait_info * cm_insert_remote_qpn(struct cm_timewait_info - *timewait_info) +static struct cm_timewait_info *cm_insert_remote_qpn(struct cm_timewait_info + *timewait_info) { struct rb_node **link = &cm.remote_qp_table.rb_node; struct rb_node *parent = NULL; @@ -645,8 +645,8 @@ static struct cm_timewait_info * cm_insert_remote_qpn(struct cm_timewait_info return NULL; } -static struct cm_id_private * cm_insert_remote_sidr(struct cm_id_private - *cm_id_priv) +static struct cm_id_private *cm_insert_remote_sidr(struct cm_id_private + *cm_id_priv) { struct rb_node **link = &cm.remote_sidr_table.rb_node; struct rb_node *parent = NULL; @@ -722,7 +722,7 @@ error: } EXPORT_SYMBOL(ib_create_cm_id); -static struct cm_work * cm_dequeue_work(struct cm_id_private *cm_id_priv) +static struct cm_work *cm_dequeue_work(struct cm_id_private *cm_id_priv) { struct cm_work *work; @@ -779,7 +779,7 @@ static void cm_cleanup_timewait(struct cm_timewait_info *timewait_info) } } -static struct cm_timewait_info * cm_create_timewait_info(__be32 local_id) +static struct cm_timewait_info *cm_create_timewait_info(__be32 local_id) { struct cm_timewait_info *timewait_info; @@ -1350,7 +1350,7 @@ static void cm_format_rej(struct cm_rej_msg *rej_msg, cm_format_mad_hdr(&rej_msg->hdr, CM_REJ_ATTR_ID, cm_id_priv->tid); rej_msg->remote_comm_id = cm_id_priv->id.remote_id; - switch(cm_id_priv->id.state) { + switch (cm_id_priv->id.state) { case IB_CM_REQ_RCVD: rej_msg->local_comm_id = 0; cm_rej_set_msg_rejected(rej_msg, CM_MSG_RESPONSE_REQ); @@ -1423,8 +1423,8 @@ unlock: spin_unlock_irq(&cm_id_priv->lock); free: cm_free_msg(msg); } -static struct cm_id_private * cm_match_req(struct cm_work *work, - struct cm_id_private *cm_id_priv) +static struct cm_id_private *cm_match_req(struct cm_work *work, + struct cm_id_private *cm_id_priv) { struct cm_id_private *listen_cm_id_priv, *cur_cm_id_priv; struct cm_timewait_info *timewait_info; @@ -2281,7 +2281,7 @@ static void cm_format_rej_event(struct cm_work *work) work->cm_event.private_data = &rej_msg->private_data; } -static struct cm_id_private * cm_acquire_rejected_id(struct cm_rej_msg *rej_msg) +static struct cm_id_private *cm_acquire_rejected_id(struct cm_rej_msg *rej_msg) { struct cm_timewait_info *timewait_info; struct cm_id_private *cm_id_priv; @@ -2291,7 +2291,7 @@ static struct cm_id_private * cm_acquire_rejected_id(struct cm_rej_msg *rej_msg) if (__be16_to_cpu(rej_msg->reason) == IB_CM_REJ_TIMEOUT) { spin_lock_irq(&cm.lock); - timewait_info = cm_find_remote_id( *((__be64 *) rej_msg->ari), + timewait_info = cm_find_remote_id(*((__be64 *) rej_msg->ari), remote_id); if (!timewait_info) { spin_unlock_irq(&cm.lock); @@ -2396,7 +2396,7 @@ int ib_send_cm_mra(struct ib_cm_id *cm_id, cm_id_priv = container_of(cm_id, struct cm_id_private, id); spin_lock_irqsave(&cm_id_priv->lock, flags); - switch(cm_id_priv->id.state) { + switch (cm_id_priv->id.state) { case IB_CM_REQ_RCVD: cm_state = IB_CM_MRA_REQ_SENT; lap_state = cm_id->lap_state; @@ -2448,7 +2448,7 @@ error2: spin_unlock_irqrestore(&cm_id_priv->lock, flags); } EXPORT_SYMBOL(ib_send_cm_mra); -static struct cm_id_private * cm_acquire_mraed_id(struct cm_mra_msg *mra_msg) +static struct cm_id_private *cm_acquire_mraed_id(struct cm_mra_msg *mra_msg) { switch (cm_mra_get_msg_mraed(mra_msg)) { case CM_MSG_RESPONSE_REQ: @@ -3270,8 +3270,7 @@ static int cm_establish(struct ib_cm_id *cm_id) cm_id_priv = container_of(cm_id, struct cm_id_private, id); spin_lock_irqsave(&cm_id_priv->lock, flags); - switch (cm_id->state) - { + switch (cm_id->state) { case IB_CM_REP_SENT: case IB_CM_MRA_REP_RCVD: cm_id->state = IB_CM_ESTABLISHED; diff --git a/drivers/infiniband/core/cm_msgs.h b/drivers/infiniband/core/cm_msgs.h index aec9c7a..8df76ae 100644 --- a/drivers/infiniband/core/cm_msgs.h +++ b/drivers/infiniband/core/cm_msgs.h @@ -172,7 +172,7 @@ static inline void cm_req_set_remote_resp_timeout(struct cm_req_msg *req_msg, static inline enum ib_qp_type cm_req_get_qp_type(struct cm_req_msg *req_msg) { u8 transport_type = (u8) (be32_to_cpu(req_msg->offset40) & 0x06) >> 1; - switch(transport_type) { + switch (transport_type) { case 0: return IB_QPT_RC; case 1: return IB_QPT_UC; default: return 0; @@ -182,7 +182,7 @@ static inline enum ib_qp_type cm_req_get_qp_type(struct cm_req_msg *req_msg) static inline void cm_req_set_qp_type(struct cm_req_msg *req_msg, enum ib_qp_type qp_type) { - switch(qp_type) { + switch (qp_type) { case IB_QPT_UC: req_msg->offset40 = cpu_to_be32((be32_to_cpu( req_msg->offset40) & diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index ae11d5c..895d88f 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -221,7 +221,8 @@ static int cma_comp_exch(struct rdma_id_private *id_priv, int ret; spin_lock_irqsave(&id_priv->lock, flags); - if ((ret = (id_priv->state == comp))) + ret = (id_priv->state == comp); + if (ret) id_priv->state = exch; spin_unlock_irqrestore(&id_priv->lock, flags); return ret; @@ -2277,7 +2278,7 @@ static int cma_connect_iw(struct rdma_id_private *id_priv, struct rdma_conn_param *conn_param) { struct iw_cm_id *cm_id; - struct sockaddr_in* sin; + struct sockaddr_in *sin; int ret; struct iw_cm_conn_param iw_param; @@ -2289,10 +2290,10 @@ static int cma_connect_iw(struct rdma_id_private *id_priv, id_priv->cm_id.iw = cm_id; - sin = (struct sockaddr_in*) &id_priv->id.route.addr.src_addr; + sin = (struct sockaddr_in *) &id_priv->id.route.addr.src_addr; cm_id->local_addr = *sin; - sin = (struct sockaddr_in*) &id_priv->id.route.addr.dst_addr; + sin = (struct sockaddr_in *) &id_priv->id.route.addr.dst_addr; cm_id->remote_addr = *sin; ret = cma_modify_qp_rtr(id_priv, conn_param); @@ -2399,9 +2400,9 @@ static int cma_accept_iw(struct rdma_id_private *id_priv, iw_param.ird = conn_param->responder_resources; iw_param.private_data = conn_param->private_data; iw_param.private_data_len = conn_param->private_data_len; - if (id_priv->id.qp) { + if (id_priv->id.qp) iw_param.qpn = id_priv->qp_num; - } else + else iw_param.qpn = conn_param->qp_num; return iw_cm_accept(id_priv->cm_id.iw, &iw_param); diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 7913b80..272a3fe 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -49,7 +49,7 @@ MODULE_LICENSE("Dual BSD/GPL"); struct ib_client_data { struct list_head list; struct ib_client *client; - void * data; + void *data; }; static LIST_HEAD(device_list); diff --git a/drivers/infiniband/core/fmr_pool.c b/drivers/infiniband/core/fmr_pool.c index 4507043..078370e 100644 --- a/drivers/infiniband/core/fmr_pool.c +++ b/drivers/infiniband/core/fmr_pool.c @@ -92,7 +92,7 @@ struct ib_fmr_pool { struct hlist_head *cache_bucket; void (*flush_function)(struct ib_fmr_pool *pool, - void * arg); + void *arg); void *flush_arg; struct task_struct *thread; @@ -149,7 +149,7 @@ static void ib_fmr_batch_release(struct ib_fmr_pool *pool) list_add_tail(&fmr->fmr->list, &fmr_list); #ifdef DEBUG - if (fmr->ref_count !=0) { + if (fmr->ref_count != 0) { printk(KERN_WARNING PFX "Unmapping FMR 0x%08x with ref count %d\n", fmr, fmr->ref_count); } @@ -161,9 +161,8 @@ static void ib_fmr_batch_release(struct ib_fmr_pool *pool) spin_unlock_irq(&pool->pool_lock); - if (list_empty(&unmap_list)) { + if (list_empty(&unmap_list)) return; - } ret = ib_unmap_fmr(&fmr_list); if (ret) @@ -306,7 +305,7 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, int bytes_per_fmr = sizeof *fmr; if (pool->cache_bucket) - bytes_per_fmr += params->max_pages_per_fmr * sizeof (u64); + bytes_per_fmr += params->max_pages_per_fmr * sizeof u64; for (i = 0; i < params->pool_size; ++i) { fmr = kmalloc(bytes_per_fmr, GFP_KERNEL); @@ -452,9 +451,8 @@ struct ib_pool_fmr *ib_fmr_pool_map_phys(struct ib_fmr_pool *pool_handle, if (fmr) { /* found in cache */ ++fmr->ref_count; - if (fmr->ref_count == 1) { + if (fmr->ref_count == 1) list_del(&fmr->list); - } spin_unlock_irqrestore(&pool->pool_lock, flags); diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index 81c9195..53f53fc 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -158,7 +158,7 @@ static void free_cm_id(struct iwcm_id_private *cm_id_priv) */ static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv) { - BUG_ON(atomic_read(&cm_id_priv->refcount)==0); + BUG_ON(atomic_read(&cm_id_priv->refcount) == 0); if (atomic_dec_and_test(&cm_id_priv->refcount)) { BUG_ON(!list_empty(&cm_id_priv->work_list)); complete(&cm_id_priv->destroy_comp); @@ -652,7 +652,7 @@ static void cm_conn_req_handler(struct iwcm_id_private *listen_id_priv, iw_cm_reject(cm_id, NULL, 0); set_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags); destroy_cm_id(cm_id); - if (atomic_read(&cm_id_priv->refcount)==0) + if (atomic_read(&cm_id_priv->refcount) == 0) free_cm_id(cm_id_priv); } @@ -857,7 +857,7 @@ static void cm_work_handler(struct work_struct *_work) set_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags); destroy_cm_id(&cm_id_priv->id); } - BUG_ON(atomic_read(&cm_id_priv->refcount)==0); + BUG_ON(atomic_read(&cm_id_priv->refcount) == 0); destroy_id = test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags); if (iwcm_deref_id(cm_id_priv)) { if (destroy_id) { diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c index 4d10421..f85a21a 100644 --- a/drivers/infiniband/core/sysfs.c +++ b/drivers/infiniband/core/sysfs.c @@ -687,7 +687,7 @@ static ssize_t show_##name(struct device *device, \ { \ return show_protocol_stat(device, attr, buf, \ offsetof(struct iw_protocol_stats, name) / \ - sizeof (u64)); \ + sizeof u64); \ } \ static DEVICE_ATTR(name, S_IRUGO, show_##name, NULL) diff --git a/drivers/infiniband/core/ud_header.c b/drivers/infiniband/core/ud_header.c index 8ec7876..f1f6da2 100644 --- a/drivers/infiniband/core/ud_header.c +++ b/drivers/infiniband/core/ud_header.c @@ -202,9 +202,8 @@ void ib_ud_header_init(int payload_bytes, IB_LRH_BYTES + IB_BTH_BYTES + IB_DETH_BYTES; - if (grh_present) { + if (grh_present) header_len += IB_GRH_BYTES; - } header->lrh.link_version = 0; header->lrh.link_next_header = From dotanba at gmail.com Fri Jul 18 01:19:26 2008 From: dotanba at gmail.com (Dotan Barak) Date: Fri, 18 Jul 2008 11:19:26 +0300 Subject: [ofa-general] ***SPAM*** [PATCH] core/iwcm: remove IB_ACCESS_LOCAL_WRITE in remote QP attributes Message-ID: <200807181119.27203.dotanba@gmail.com> IB_ACCESS_LOCAL_WRITE was removed for qp.qp_access_flags because this attribute is relevant only for remote permissions. Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index 81c9195..53f53fc 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -942,8 +942,7 @@ static int iwcm_init_qp_init_attr(struct iwcm_id_private *cm_id_priv, case IW_CM_STATE_CONN_RECV: case IW_CM_STATE_ESTABLISHED: *qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS; - qp_attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE | - IB_ACCESS_REMOTE_WRITE| + qp_attr->qp_access_flags = IB_ACCESS_REMOTE_WRITE| IB_ACCESS_REMOTE_READ; ret = 0; break; From kliteyn at dev.mellanox.co.il Fri Jul 18 08:06:41 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Fri, 18 Jul 2008 18:06:41 +0300 Subject: [ofa-general] OpenSM "Dead end on path to LID" In-Reply-To: <487EA1CC.7060308@noaa.gov> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> Message-ID: <4880B181.2090400@dev.mellanox.co.il> Hi Nathan, Nathan Dauchy wrote: > > Looking through osm.log a bit more, I also found a handful of errors > like these: > > Jul 17 01:31:29 345329 [46E0A940] 0x01 -> > __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for > node 0x000002c900000048(MT47396 Infiniscale-III Mellanox Technologies) > port 14. Adding to light sweep sampling list > Jul 17 01:31:29 345340 [46E0A940] 0x01 -> Directed Path Dump of 4 hop path: > Path = 0,1,20,7,15 > Jul 17 01:31:29 345381 [46E0A940] 0x01 -> > __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for > node 0x000002c900000049(MT47396 Infiniscale-III Mellanox Technologies) > port 15. Adding to light sweep sampling list > Jul 17 01:31:29 345390 [46E0A940] 0x01 -> Directed Path Dump of 3 hop path: > Path = 0,1,22,11 > > Does that indicate a problem as well? This explains why ibdiagnet couldn't query port counters. OpenSM couldn't discover what's behind these ports, so it didn't configure routing tables for the undiscovered nodes. Ibdiagnet could discover them. It queries port counters by their LIDs, but switches don't have these LIDs in the routing tables. > Unknown remote side for node 0x000002c900000049(MT47396 > Infiniscale-III Mellanox Technologies) port 15 What is the remote side of this port? HCA? Switch? If it's HCA, does its host run some heavy application? I understand you already increased transaction time. Please try limiting SMPs on the wire - in opensm.conf file, set max_wire_smps to 1 (you probably have 4). You can also run opensm with '-maxsmps 1' command line argument. -- Yevgeny > Thanks, > Nathan > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From Nathan.Dauchy at noaa.gov Fri Jul 18 09:34:34 2008 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Fri, 18 Jul 2008 10:34:34 -0600 Subject: [ofa-general] OpenSM "Dead end on path to LID" In-Reply-To: <4880B181.2090400@dev.mellanox.co.il> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> Message-ID: <4880C61A.6010000@noaa.gov> Hi Yevgeny, thanks for your response, Yevgeny Kliteynik wrote: > Hi Nathan, > > Nathan Dauchy wrote: >> >> Looking through osm.log a bit more, I also found a handful of errors >> like these: >> >> Jul 17 01:31:29 345329 [46E0A940] 0x01 -> >> __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for >> node 0x000002c900000048(MT47396 Infiniscale-III Mellanox Technologies) >> port 14. Adding to light sweep sampling list >> Jul 17 01:31:29 345340 [46E0A940] 0x01 -> Directed Path Dump of 4 hop >> path: >> Path = 0,1,20,7,15 >> Jul 17 01:31:29 345381 [46E0A940] 0x01 -> >> __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for >> node 0x000002c900000049(MT47396 Infiniscale-III Mellanox Technologies) >> port 15. Adding to light sweep sampling list >> Jul 17 01:31:29 345390 [46E0A940] 0x01 -> Directed Path Dump of 3 hop >> path: >> Path = 0,1,22,11 >> >> Does that indicate a problem as well? > > This explains why ibdiagnet couldn't query port counters. > OpenSM couldn't discover what's behind these ports, so it > didn't configure routing tables for the undiscovered nodes. > Ibdiagnet could discover them. It queries port counters by > their LIDs, but switches don't have these LIDs in the > routing tables. Thanks, that makes sense. >> Unknown remote side for node 0x000002c900000049(MT47396 >> Infiniscale-III Mellanox Technologies) port 15 > > What is the remote side of this port? HCA? Switch? > If it's HCA, does its host run some heavy application? The remote side of that port is a "spine" switch. The remote side of the other example error message is a "clos"/"edge" switch. I guess I should provide some info on our IB network topology, since it may be a little unique and contributing to the problem... The Infiniband network consists of 3 layers of switches. All switches are 24-port Flextronics DDR switches (FX-X4300??). We can refer to the layers as "Edge" (clos), "Spine", and "Root" (aggregation). The network is divided into 3 "subtrees", joined by the (2) Root Aggregation switches. We can refer to the subtrees as A, B, and C. Subtree A: 22 Edge switches 17 SDR Hosts per Edge switch 6 Spine switches Each Edge switch has an uplink to each Spine Each Spine switch has an uplink to each Root Subtree B: 22 Edge switches 12 DDR Hosts per Edge switch 9 Spine switches Each Edge switch has an uplink to each Spine Each Spine switch has an uplink to each Root Subtree C: 4 Edge switches Edge switches are configured with 9 ports as 3 logical 12x links Up to 15 SDR/DDR Hosts per Edge switch 3 Spine switches Spines are configured with all 24 ports as 8 logical 12x links Each Edge switch has an uplink (3 cables) to each Spine Each Spine switch has an uplink (3 cables) to each Root Aggregation: 2 Root switches Configured with 9 physical ports as 3 logical 12x ports 6 links to Subtree A (each) 9 links to Subtree B 3 links (9 cables) to Subtree C The Flextronics switches are 24-port DDR switches (FX-X4300??) using Mellanox Part MTS2400 (Silicon MT47396). They are are burned with firmware version "fw-47396-1.0.0", using the "M24D0601A.INI" file, with changes only to the "[LinkWidthSupp]" section. We downloaded the firmware from: http://www.mellanox.com/support/switch_firmware_table.php So, the example "Unknown remote side" messages from above are: System B Edge -> System B Spine System A Spine -> System A Edge > I understand you already increased transaction time. > Please try limiting SMPs on the wire - in opensm.conf > file, set max_wire_smps to 1 (you probably have 4). > You can also run opensm with '-maxsmps 1' command line > argument. Interesting! I believe MAXSMPS was originally set to 0 (unlimited), based on duplicating the config file from an older SM setup. I reduced it to 32 when we saw some IB errors on standalone System B. I'm afraid I don't have documentation on what those problems were, but I don't recall seeing the exact same symptoms. I think it was MAD timeout error messages that prompted me to change the MAXSMPS value. We will be able to test any fixes during a scheduled system downtime on 7/24. At that point, do you recommend trying MAXSMPS=4? (I assume the tradeoff of number of SMPs is discovery speed vs. stability. Yes?) If that doesn't work, what else should we be prepared to try, or what other debugging information would be helpful to gather? For the record, other steps we are considering: * Latest OpenSM code * Upgrading firmware on all IB switches * Changing topology to remove the 12X links (ugh!) Thanks much, Nathan From Francoise-inizzab at newvisionvideo.com Fri Jul 18 11:15:23 2008 From: Francoise-inizzab at newvisionvideo.com (Francoise) Date: Fri, 18 Jul 2008 13:15:23 -0500 Subject: [ofa-general] World record love making Message-ID: Why bikinis look like what they are today, and why not any skimpier. http://canadawax.com/news.html -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From jsquyres at cisco.com Fri Jul 18 14:23:23 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Fri, 18 Jul 2008 17:23:23 -0400 Subject: [ofa-general] IBCM and ordering of OpenFabrics devices Message-ID: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> (re-post from http://www.open-mpi.org/community/lists/devel/2008/07/4371.php ; I got no reply on the OMPI mailing list) I have a case where ib_cm_open_device() is failing for an odd reason: I have 12 servers that contain both HCAs and iWARP NICs. In most cases, everything is fine. But one one of these servers, IBCM refuses to work -- ib_cm_open_device() fails with the following: libibcm: unable to open /dev/infiniband/ucm1 Looking closer, this device does, indeed, exist: [4:01] svbu-mpi044:~/mpi % ls -l /dev/infiniband/ucm* crw-rw-rw- 1 root root 231, 224 Jul 16 04:30 /dev/infiniband/ucm0 crw-rw-rw- 1 root root 231, 225 Jul 16 04:30 /dev/infiniband/ucm1 [4:08] svbu-mpi044:~/mpi % Granted; I had to create these devices manually because they are not created automatically for me upon boot in RHEL4U4 and U6. These device major/minor numbers work fine for me on all my other servers. So what's different between the 11 machines that work and the 1 that doesn't? It seems that the kernel ordering of devices is what is different. On most of the machines: [4:10] svbu-mpi045:~ % ibv_devinfo hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0002:c903:0000:036c sys_image_guid: 0002:c903:0000:036f vendor_id: 0x02c9 vendor_part_id: 25418 hw_ver: 0xA0 board_id: MT_04A0110002 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 2 port_lid: 7 port_lmc: 0x00 port: 2 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 2 port_lid: 52 port_lmc: 0x00 hca_id: nes0 node_guid: 0012:5502:63c0:0000 sys_image_guid: 0012:5502:63c0:0000 vendor_id: 0x0000 vendor_part_id: 0 hw_ver: 0x5 board_id: NES020 Board ID phys_port_cnt: 1 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 1 port_lmc: 0x00 But on this one problematic machine: 4:10] svbu-mpi044:~/mpi % ibv_devinfo hca_id: nes0 node_guid: 0012:5502:63b8:0000 sys_image_guid: 0012:5502:63b8:0000 vendor_id: 0x0000 vendor_part_id: 0 hw_ver: 0x5 board_id: NES020 Board ID phys_port_cnt: 1 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 1 port_lmc: 0x00 hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0002:c903:0000:03b0 sys_image_guid: 0002:c903:0000:03b3 vendor_id: 0x02c9 vendor_part_id: 25418 hw_ver: 0xA0 board_id: MT_04A0110002 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 2 port_lid: 6 port_lmc: 0x00 port: 2 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 2 port_lid: 136 port_lmc: 0x00 Notice that the ordering is different. So I'm not enough of a kernel guy to know where the problem is: 1. Technically, mlx4_0 is the first IB device. Should it therefore be using ucm0? I.e., is libibcm wrong for trying to use ucm1? (note that OMPI's openib BTL is currently replicating the logic from libibcm to check for the Right ucm* file so that we can silently fail before ib_cm_open_device() fails with a warning message -- so if libibcm's logic to find the Right ucm* file changes, we'll also need to change MPI's logic to mirror it. OMPI's logic becomes moot in newer libibcm versions where Sean removed the warning message, though). 2. Or are my major/minor numbers incorrect for the devices that I created manually? If the major/minor device numbers were created by the OS upon bootup (as they should be -- there's an open OpenFabrics bugzilla ticket about this), would they be correct? -- Jeff Squyres Cisco Systems From kliteyn at dev.mellanox.co.il Fri Jul 18 14:54:14 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sat, 19 Jul 2008 00:54:14 +0300 Subject: [ofa-general] OpenSM "Dead end on path to LID" In-Reply-To: <4880C61A.6010000@noaa.gov> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> <4880C61A.6010000@noaa.gov> Message-ID: <48811106.9030906@dev.mellanox.co.il> Hi Nathan, Nathan Dauchy wrote: > Hi Yevgeny, thanks for your response, > > Yevgeny Kliteynik wrote: >> Hi Nathan, >> >> Nathan Dauchy wrote: >>> Looking through osm.log a bit more, I also found a handful of errors >>> like these: >>> >>> Jul 17 01:31:29 345329 [46E0A940] 0x01 -> >>> __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for >>> node 0x000002c900000048(MT47396 Infiniscale-III Mellanox Technologies) >>> port 14. Adding to light sweep sampling list >>> Jul 17 01:31:29 345340 [46E0A940] 0x01 -> Directed Path Dump of 4 hop >>> path: >>> Path = 0,1,20,7,15 >>> Jul 17 01:31:29 345381 [46E0A940] 0x01 -> >>> __osm_state_mgr_light_sweep_start: ERR 0108: Unknown remote side for >>> node 0x000002c900000049(MT47396 Infiniscale-III Mellanox Technologies) >>> port 15. Adding to light sweep sampling list >>> Jul 17 01:31:29 345390 [46E0A940] 0x01 -> Directed Path Dump of 3 hop >>> path: >>> Path = 0,1,22,11 >>> >>> Does that indicate a problem as well? >> This explains why ibdiagnet couldn't query port counters. >> OpenSM couldn't discover what's behind these ports, so it >> didn't configure routing tables for the undiscovered nodes. >> Ibdiagnet could discover them. It queries port counters by >> their LIDs, but switches don't have these LIDs in the >> routing tables. > > Thanks, that makes sense. > >>> Unknown remote side for node 0x000002c900000049(MT47396 >>> Infiniscale-III Mellanox Technologies) port 15 >> What is the remote side of this port? HCA? Switch? >> If it's HCA, does its host run some heavy application? > > The remote side of that port is a "spine" switch. The remote side of > the other example error message is a "clos"/"edge" switch. > > I guess I should provide some info on our IB network topology, since it > may be a little unique and contributing to the problem... > > The Infiniband network consists of 3 layers of switches. All switches > are 24-port Flextronics DDR switches (FX-X4300??). We can refer to the > layers as "Edge" (clos), "Spine", and "Root" (aggregation). The network > is divided into 3 "subtrees", joined by the (2) Root Aggregation > switches. We can refer to the subtrees as A, B, and C. > > Subtree A: > 22 Edge switches > 17 SDR Hosts per Edge switch > 6 Spine switches > Each Edge switch has an uplink to each Spine > Each Spine switch has an uplink to each Root > > Subtree B: > 22 Edge switches > 12 DDR Hosts per Edge switch > 9 Spine switches > Each Edge switch has an uplink to each Spine > Each Spine switch has an uplink to each Root > > Subtree C: > 4 Edge switches > Edge switches are configured with 9 ports as 3 logical 12x links > Up to 15 SDR/DDR Hosts per Edge switch > 3 Spine switches > Spines are configured with all 24 ports as 8 logical 12x links > Each Edge switch has an uplink (3 cables) to each Spine > Each Spine switch has an uplink (3 cables) to each Root > > Aggregation: > 2 Root switches > Configured with 9 physical ports as 3 logical 12x ports > 6 links to Subtree A (each) > 9 links to Subtree B > 3 links (9 cables) to Subtree C > > The Flextronics switches are 24-port DDR switches (FX-X4300??) using > Mellanox Part MTS2400 (Silicon MT47396). They are are burned with > firmware version "fw-47396-1.0.0", using the "M24D0601A.INI" file, with > changes only to the "[LinkWidthSupp]" section. We downloaded the > firmware from: http://www.mellanox.com/support/switch_firmware_table.php > > So, the example "Unknown remote side" messages from above are: > System B Edge -> System B Spine > System A Spine -> System A Edge > > >> I understand you already increased transaction time. >> Please try limiting SMPs on the wire - in opensm.conf >> file, set max_wire_smps to 1 (you probably have 4). >> You can also run opensm with '-maxsmps 1' command line >> argument. > > Interesting! > > I believe MAXSMPS was originally set to 0 (unlimited), based on > duplicating the config file from an older SM setup. I reduced it to 32 > when we saw some IB errors on standalone System B. I'm afraid I don't > have documentation on what those problems were, but I don't recall > seeing the exact same symptoms. I think it was MAD timeout error > messages that prompted me to change the MAXSMPS value. You probably have the timeout errors in the osm log now as well. These errors are followed by the "unknown remote side" messages, which means that osm didn't get response to some MADs (timeout). > We will be able to test any fixes during a scheduled system downtime on > 7/24. At that point, do you recommend trying MAXSMPS=4? In ofed 1.3.1, the default conf file value for maxsmps should be 4. If you're using 0, then I'm pretty sure that this is what causing the problems. Try 4 - it should be fine. > (I assume the > tradeoff of number of SMPs is discovery speed vs. stability. Yes?) In short - yes. On big clusters there could be problems with VL15 traffic overflowing VL15 buffers in the switches. Increasing transaction time doesn't help, as the packets are dropped, not delayed. > If that doesn't work, what else should we be prepared to try, or what > other debugging information would be helpful to gather? > > For the record, other steps we are considering: > * Latest OpenSM code > * Upgrading firmware on all IB switches > * Changing topology to remove the 12X links (ugh!) Hope that there won't be a need for other steps. -- Yevgeny > > Thanks much, > Nathan > From matthias at sgi.com Fri Jul 18 22:29:43 2008 From: matthias at sgi.com (Matthias Blankenhaus) Date: Fri, 18 Jul 2008 22:29:43 -0700 (PDT) Subject: [ofa-general] Howto identify master SM Message-ID: Howdy ! I was not able to find a tool that lets me identify the master or standby on a fabric. I have used ibsysstat, smpquery and smpdump, but that did not get me any closer. How can I identify a master / standby SM from CLI ? Thanx, Matthias From 53khalid at arabian-pipes.com Fri Jul 18 22:33:03 2008 From: 53khalid at arabian-pipes.com (john raymond) Date: Sat, 19 Jul 2008 05:33:03 +0000 Subject: [ofa-general] to general Message-ID: <000701c8e96f$022056a0$574cb9b3@yktuxe> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.saqwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From cindy.derocco at natoil.com Fri Jul 18 23:04:04 2008 From: cindy.derocco at natoil.com (viagar icalis ) Date: Sat, 19 Jul 2008 06:04:04 +0000 Subject: [ofa-general] 95% off for general Message-ID: <42821.ruben@aidan> Hello, make a right choice, buy your pills from the most reliable shop. http://www.zsoka.de/getinfo.html Code #ZICQ garrard steven From kliteyn at dev.mellanox.co.il Sat Jul 19 00:55:19 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sat, 19 Jul 2008 10:55:19 +0300 Subject: [ofa-general] Howto identify master SM In-Reply-To: References: Message-ID: <48819DE7.40904@dev.mellanox.co.il> Matthias Blankenhaus wrote: > Howdy ! > > I was not able to find a tool that lets me > identify the master or standby on a fabric. I have used > ibsysstat, smpquery and smpdump, but that did not get me any > closer. > > How can I identify a master / standby SM from CLI ? sminfo -- Yevgeny > > Thanx, > Matthias > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From vlad at lists.openfabrics.org Sat Jul 19 02:50:07 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sat, 19 Jul 2008 02:50:07 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080719-0200 daily build status Message-ID: <20080719095007.294D9E60849@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Failed: Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: include/asm/apic.h:47: warning: value computed is not used /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c: In function 'ipath_diagpkt_write': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.c:473: error: implicit declaration of function '__iowrite32_copy' make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080719-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From dotanba at gmail.com Sat Jul 19 04:11:06 2008 From: dotanba at gmail.com (Dotan Barak) Date: Sat, 19 Jul 2008 13:11:06 +0200 Subject: [ofa-general] Howto identify master SM In-Reply-To: <48819DE7.40904@dev.mellanox.co.il> References: <48819DE7.40904@dev.mellanox.co.il> Message-ID: <4881CBCA.2020100@gmail.com> Yevgeny Kliteynik wrote: > Matthias Blankenhaus wrote: >> Howdy ! >> >> I was not able to find a tool that lets me >> identify the master or standby on a fabric. I have used >> ibsysstat, smpquery and smpdump, but that did not get me any >> closer. >> >> How can I identify a master / standby SM from CLI ? > > sminfo Another way will be to use ibv_devinfo and get the sm_lid in the port properties. Dotan > > -- Yevgeny > >> >> Thanx, >> Matthias From 15i.dini at provincia.vt.it Sat Jul 19 01:55:42 2008 From: 15i.dini at provincia.vt.it (flinn marlin) Date: Sat, 19 Jul 2008 08:55:42 +0000 Subject: [ofa-general] pay off debt Message-ID: <000501c8e98c$02f6fd32$264791a6@nmeyyjb> Do Not consolidate your debt EliminateE it! Legally ELIMINATE your creditt card and other unsecured debt * WITHOUT ever making another payment to your creditors * WITHOUT it affecting your credit long-term * WITHOUT confrontation Visit www.drymisted[DOT]com (copy this link and replace "[DOT]" to ".") This IS NOT: * Bankruptcy * Consolidation * Or refinancing of any kind Visit here to learn how www.drymisted[DOT]com (copy this link and replace "[DOT]" to ".") * Must have a minimum of $10K in combined household unsecured debt to apply. * Must be a US resident. From ogerlitz at voltaire.com Sat Jul 19 23:35:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 20 Jul 2008 09:35:38 +0300 (IDT) Subject: [ofa-general] [PATCH] ib/ipoib : print error code of ib_sa path lookup call Message-ID: print the return code of ib_sa_path_rec_get to ease diagnosing errors Signed-off-by: Or Gerlitz Index: infiniband/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- infiniband.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ infiniband/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -548,7 +548,7 @@ static int path_rec_start(struct net_dev path_rec_completion, path, &path->query); if (path->query_id < 0) { - ipoib_warn(priv, "ib_sa_path_rec_get failed\n"); + ipoib_warn(priv, "ib_sa_path_rec_get failed ret=%d\n",path->query_id); path->query = NULL; return path->query_id; } From 17thomas at trinity-health.org Sat Jul 19 21:48:30 2008 From: 17thomas at trinity-health.org (dav caryl) Date: Sun, 20 Jul 2008 04:48:30 +0000 Subject: [ofa-general] to general Message-ID: <000501c8ea32$04a3a2be$5c150689@gesio> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.saqwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From vlad at lists.openfabrics.org Sun Jul 20 02:50:44 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sun, 20 Jul 2008 02:50:44 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080720-0200 daily build status Message-ID: <20080720095044.50BCDE6023C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Failed: Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080720-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Sun Jul 20 03:43:24 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 20 Jul 2008 13:43:24 +0300 (IDT) Subject: [ofa-general] reloading/applying configuration changes by opensm Message-ID: For some reason... my thinking was that each time opensm does heavy sweep it reads the cache file and if something was changed there from the previous sweep, apply this change (eg LMC was 0 and is now 1), is it correct? Practically, I see that it doesn't work this way, i.e I run opensm, then sends it a SIGHUP which indeed seems to cause a heavy sweep as noted in the man page, but the LMC change I made in the cache file doesn't apply by the sm to the fabric. Maybe this is new feature for which I need to use the master code ? Or. From pbmb at bma-ca.com Sun Jul 20 06:38:36 2008 From: pbmb at bma-ca.com (Lara Ramirez) Date: Sun, 20 Jul 2008 16:38:36 +0300 Subject: [ofa-general] Re: Message-ID: <536975333.79809332102227@bma-ca.com> IPTV – Die Zukunft des Fernsehens hat begonnen IPTV – The future for television is now To receive your favourite tv channelworldwide – IPTV does it. Not on your tv screen, but on your tv! And maxx-tv AGis one of the first-movers in this growing market. Due to the company, they will even start toenter the world’s largest tv market USA in Fall 2008. German program for morethan 40 million German-speaking people. And on top: The costs of the maketingcampaign are covered by the ste-top-box producer! This given, we recommend: BUY (SPECULATIVE) Maxx-TV AGWKN: A0M0KX, Symbol: M55FrankfurtStock ExchangeShare price: ? 0.186 months-target: ? 0.35 / 0.45 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronli.voltaire at gmail.com Sun Jul 20 09:11:01 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Sun, 20 Jul 2008 19:11:01 +0300 Subject: [ofa-general] [PATCH 0/12] new verbs: create_qp_expanded Message-ID: <3b5e77ad0807200911i1935e922r43b060c3ca44cac7@mail.gmail.com> This series of patches add support for a new verb: create_qp_expanded, which now takes another argument in addition: creation flags. These patches are based on Jack Morgenstein 12 patches for the support in XRC. Ron From ronli at voltaire.com Sun Jul 20 12:14:28 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 20 Jul 2008 19:14:28 +0000 (UTC) Subject: [ofa-general] [PATCH 1/3] libibverbs : add new verbs: create_qp_expanded Message-ID: 1. This patch adds a new capability flags to the enum ibv_device_cap_flags IBV_DEVICE_BLOCK_MULTICAST_LOOPBACK. which implies that the device is capable of blocking multicast loopback packets. 2. This patch also adds a new verb to the libibverbs: struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr, uint32_t create_flags); which works similarly to ibv_create_qp, except for that it now takes another argument: uint32_t create_flags these creation flags should be aligned with those in the ib_verbs.h in the kernel. 3. New creation flags added: IBV_QP_CREATE_IPOIB_UD_LSO, IBV_MULTICAST_LOOPBACK_BLOCK Signed-off-by: Ron Livne diff --git a/include/infiniband/driver.h b/include/infiniband/driver.h index f8138ef..47ec759 100644 --- a/include/infiniband/driver.h +++ b/include/infiniband/driver.h @@ -155,6 +155,11 @@ int ibv_cmd_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int ibv_cmd_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + uint32_t create_flags, + struct ibv_create_qp *cmd, size_t cmd_size, + struct ibv_create_qp_resp *resp, size_t resp_size); /* * sysfs helper functions diff --git a/include/infiniband/kern-abi.h b/include/infiniband/kern-abi.h index 8b5cd9a..bbd8534 100644 --- a/include/infiniband/kern-abi.h +++ b/include/infiniband/kern-abi.h @@ -94,6 +94,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, IB_USER_VERBS_CMD_REG_XRC_RCV_QP, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED, }; /* @@ -455,7 +456,7 @@ struct ibv_create_qp { __u8 sq_sig_all; __u8 qp_type; __u8 is_srq; - __u8 reserved; + __u8 create_flags; __u64 driver_data[0]; }; @@ -945,6 +946,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP_V2 = -1, IB_USER_VERBS_CMD_REG_XRC_RCV_QP_V2 = -1, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED_V2 = -1, }; struct ibv_destroy_cq_v1 { diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index 2865d5c..25a7422 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -93,7 +93,8 @@ enum ibv_device_cap_flags { IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, IBV_DEVICE_SRQ_RESIZE = 1 << 13, IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, - IBV_DEVICE_XRC = 1 << 20 + IBV_DEVICE_XRC = 1 << 20, + IBV_DEVICE_BLOCK_MULTICAST_LOOPBACK = 1 << 22 }; enum ibv_atomic_cap { @@ -631,6 +632,17 @@ enum { IBV_SYSFS_PATH_MAX = 256 }; +/* + This enum must be alligned with ib_qp_create_flags + in include/rdma/ib_verbs.h +*/ +enum ibv_qp_create_flags { + IBV_QP_CREATE_IPOIB_UD_LSO = 1 << 0, + IBV_MULTICAST_LOOPBACK_BLOCK = 1 << 1 +}; + + + struct ibv_device { struct ibv_device_ops ops; enum ibv_node_type node_type; @@ -668,6 +680,9 @@ struct ibv_more_ops { uint32_t xrc_qp_num); int (*unreg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); + struct ibv_qp * (*create_qp_expanded)(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags); }; @@ -1029,6 +1044,13 @@ struct ibv_qp *ibv_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); /** + * ibv_create_qp_expanded - Create a queue pair with creation flags. + */ +struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags); + +/** * ibv_modify_qp - Modify a queue pair. */ int ibv_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, diff --git a/src/cmd.c b/src/cmd.c index cedf55e..802f3ef 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -642,7 +642,68 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? (attr->xrc_domain ? attr->xrc_domain->handle : 0) : (attr->srq ? attr->srq->handle : 0); - cmd->reserved = 0; + cmd->create_flags = 0; + + if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) + return errno; + + VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); + + qp->handle = resp->qp_handle; + qp->qp_num = resp->qpn; + qp->context = pd->context; + + if (abi_ver > 3) { + attr->cap.max_recv_sge = resp->max_recv_sge; + attr->cap.max_send_sge = resp->max_send_sge; + attr->cap.max_recv_wr = resp->max_recv_wr; + attr->cap.max_send_wr = resp->max_send_wr; + attr->cap.max_inline_data = resp->max_inline_data; + } + + if (abi_ver == 4) { + struct ibv_create_qp_resp_v4 *resp_v4 = + (struct ibv_create_qp_resp_v4 *) resp; + + memmove((void *) resp + sizeof *resp, + (void *) resp_v4 + sizeof *resp_v4, + resp_size - sizeof *resp); + } else if (abi_ver <= 3) { + struct ibv_create_qp_resp_v3 *resp_v3 = + (struct ibv_create_qp_resp_v3 *) resp; + + memmove((void *) resp + sizeof *resp, + (void *) resp_v3 + sizeof *resp_v3, + resp_size - sizeof *resp); + } + + return 0; +} + +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + uint32_t create_flags, + struct ibv_create_qp *cmd, size_t cmd_size, + struct ibv_create_qp_resp *resp, size_t resp_size) +{ + IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP, resp, resp_size); + + cmd->create_flags = create_flags; + cmd->user_handle = (uintptr_t) qp; + cmd->pd_handle = pd->handle; + cmd->send_cq_handle = attr->send_cq->handle; + cmd->recv_cq_handle = attr->recv_cq->handle; + cmd->max_send_wr = attr->cap.max_send_wr; + cmd->max_recv_wr = attr->cap.max_recv_wr; + cmd->max_send_sge = attr->cap.max_send_sge; + cmd->max_recv_sge = attr->cap.max_recv_sge; + cmd->max_inline_data = attr->cap.max_inline_data; + cmd->sq_sig_all = attr->sq_sig_all; + cmd->qp_type = attr->qp_type; + cmd->is_srq = !!attr->srq; + cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? + (attr->xrc_domain ? attr->xrc_domain->handle : 0) : + (attr->srq ? attr->srq->handle : 0); if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; diff --git a/src/libibverbs.map b/src/libibverbs.map index fce6965..a042e31 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -112,4 +112,7 @@ IBVERBS_1.1 { ibv_port_state_str; ibv_event_type_str; ibv_wc_status_str; + + ibv_create_qp_expanded; + ibv_cmd_create_qp_expanded; } IBVERBS_1.0; diff --git a/src/verbs.c b/src/verbs.c index 4bfeec2..2e56eb7 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -449,6 +449,32 @@ struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, } default_symver(__ibv_create_qp, ibv_create_qp); +struct ibv_qp *__ibv_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags) +{ + struct ibv_qp *qp = pd->context->more_ops->create_qp_expanded(pd, qp_init_attr, create_flags); + + if (qp) { + qp->context = pd->context; + qp->qp_context = qp_init_attr->qp_context; + qp->pd = pd; + qp->send_cq = qp_init_attr->send_cq; + qp->recv_cq = qp_init_attr->recv_cq; + qp->srq = qp_init_attr->srq; + qp->qp_type = qp_init_attr->qp_type; + qp->state = IBV_QPS_RESET; + qp->events_completed = 0; + qp->xrc_domain = qp_init_attr->qp_type == IBV_QPT_XRC ? + qp_init_attr->xrc_domain : NULL; + pthread_mutex_init(&qp->mutex, NULL); + pthread_cond_init(&qp->cond, NULL); + } + + return qp; +} +default_symver(__ibv_create_qp_expanded, ibv_create_qp_expanded); + int __ibv_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, enum ibv_qp_attr_mask attr_mask, struct ibv_qp_init_attr *init_attr) From ronli at voltaire.com Sun Jul 20 12:15:21 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 20 Jul 2008 19:15:21 +0000 (UTC) Subject: [ofa-general] [PATCH 2/3] libmlx4: add support for create_qp_expanded Message-ID: Adds support for the new verb: create_qp_expanded in libmlx4. Signed-off-by: Ron Livne diff --git a/src/mlx4.c b/src/mlx4.c index 1f23144..bfcb7ec 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -79,6 +79,7 @@ static struct ibv_more_ops mlx4_more_ops = { .query_xrc_rcv_qp = mlx4_query_xrc_rcv_qp, .reg_xrc_rcv_qp = mlx4_reg_xrc_rcv_qp, .unreg_xrc_rcv_qp = mlx4_unreg_xrc_rcv_qp, + .create_qp_expanded = mlx4_create_qp_expanded, #endif }; #endif diff --git a/src/mlx4.h b/src/mlx4.h index a2e3aec..7ed3ac8 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -427,6 +427,9 @@ int mlx4_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags); #endif diff --git a/src/verbs.c b/src/verbs.c index b1bad3c..9a99658 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -825,4 +825,115 @@ int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, return ibv_cmd_unreg_xrc_rcv_qp(xrc_domain, xrc_qp_num); } +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags) +{ + struct mlx4_create_qp cmd; + struct ibv_create_qp_resp resp; + struct mlx4_qp *qp; + int ret; + + /* Sanity check QP size before proceeding */ + if (attr->cap.max_send_wr > 65536 || + attr->cap.max_recv_wr > 65536 || + attr->cap.max_send_sge > 64 || + attr->cap.max_recv_sge > 64 || + attr->cap.max_inline_data > 1024) + return NULL; + + qp = malloc(sizeof *qp); + if (!qp) + return NULL; + + mlx4_calc_sq_wqe_size(&attr->cap, attr->qp_type, qp); + + /* + * We need to leave 2 KB + 1 WQE of headroom in the SQ to + * allow HW to prefetch. + */ + qp->sq_spare_wqes = (2048 >> qp->sq.wqe_shift) + 1; + qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); + qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); + + if (attr->srq || attr->qp_type == IBV_QPT_XRC) + attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; + else { + if (attr->cap.max_recv_sge < 1) + attr->cap.max_recv_sge = 1; + if (attr->cap.max_recv_wr < 1) + attr->cap.max_recv_wr = 1; + } + + if (mlx4_alloc_qp_buf(pd, &attr->cap, attr->qp_type, qp)) + goto err; + + mlx4_init_qp_indices(qp); + + if (pthread_spin_init(&qp->sq.lock, PTHREAD_PROCESS_PRIVATE) || + pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) + goto err_free; + + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { + qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); + if (!qp->db) + goto err_free; + + *qp->db = 0; + } + + cmd.buf_addr = (uintptr_t) qp->buf.buf; + if (attr->srq || attr->qp_type == IBV_QPT_XRC) + cmd.db_addr = 0; + else + cmd.db_addr = (uintptr_t) qp->db; + cmd.log_sq_stride = qp->sq.wqe_shift; + for (cmd.log_sq_bb_count = 0; + qp->sq.wqe_cnt > 1 << cmd.log_sq_bb_count; + ++cmd.log_sq_bb_count) + ; /* nothing */ + cmd.sq_no_prefetch = 0; + + ret = ibv_cmd_create_qp_expanded(pd, &qp->ibv_qp, attr, create_flags, + &cmd.ibv_cmd, sizeof cmd, + &resp, sizeof resp); + if (ret) + goto err_rq_db; + + ret = mlx4_store_qp(to_mctx(pd->context), qp->ibv_qp.qp_num, qp); + if (ret) + goto err_destroy; + + qp->rq.wqe_cnt = qp->rq.max_post = attr->cap.max_recv_wr; + qp->rq.max_gs = attr->cap.max_recv_sge; + mlx4_set_sq_sizes(qp, &attr->cap, attr->qp_type); + + qp->doorbell_qpn = htonl(qp->ibv_qp.qp_num << 8); + if (attr->sq_sig_all) + qp->sq_signal_bits = htonl(MLX4_WQE_CTRL_CQ_UPDATE); + else + qp->sq_signal_bits = 0; + + return &qp->ibv_qp; + +err_destroy: + ibv_cmd_destroy_qp(&qp->ibv_qp); + +err_rq_db: + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); + +err_free: + free(qp->sq.wrid); + if (qp->rq.wqe_cnt) + free(qp->rq.wrid); + mlx4_free_buf(&qp->buf); + +err: + free(qp); + + return NULL; +} + + #endif From ronli at voltaire.com Sun Jul 20 12:16:01 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 20 Jul 2008 19:16:01 +0000 (UTC) Subject: [ofa-general] [PATCH 3/3] ib/uverbs: add support for create_qp_expanded in uverbs Message-ID: This patch adds support for create_qp_expanded to the uverbs. It uses the reserved bitmap in ib_uverbs_create_qp to transfer the new creation flags from the user space to the kernel. Signed-off-by: Ron Livne diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index b55f0d7..ae9f9a8 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -214,6 +214,7 @@ IB_UVERBS_DECLARE_CMD(modify_xrc_rcv_qp); IB_UVERBS_DECLARE_CMD(query_xrc_rcv_qp); IB_UVERBS_DECLARE_CMD(reg_xrc_rcv_qp); IB_UVERBS_DECLARE_CMD(unreg_xrc_rcv_qp); +IB_UVERBS_DECLARE_CMD(create_qp_expanded); #endif /* UVERBS_H */ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 4402a07..49da4c5 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1078,7 +1078,6 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, goto err_put; } - attr.create_flags = 0; attr.event_handler = ib_uverbs_qp_event_handler; attr.qp_context = file; attr.send_cq = scq; @@ -1087,7 +1086,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; attr.xrc_domain = xrcd; - attr.create_flags = 0; + attr.create_flags = cmd.reserved; attr.cap.max_send_wr = cmd.max_send_wr; attr.cap.max_recv_wr = cmd.max_recv_wr; @@ -1184,6 +1183,161 @@ err_put: return ret; } +ssize_t ib_uverbs_create_qp_expanded(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_create_qp cmd; + struct ib_uverbs_create_qp_resp resp; + struct ib_udata udata; + struct ib_uqp_object *obj; + struct ib_pd *pd; + struct ib_cq *scq, *rcq; + struct ib_srq *srq; + struct ib_qp *qp; + struct ib_qp_init_attr attr; + struct ib_xrcd *xrcd; + struct ib_uobject *xrcd_uobj; + int ret; + + if (out_len < sizeof resp) + return -ENOSPC; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + INIT_UDATA(&udata, buf + sizeof cmd, + (unsigned long) cmd.response + sizeof resp, + in_len - sizeof cmd, out_len - sizeof resp); + + obj = kmalloc(sizeof *obj, GFP_KERNEL); + if (!obj) + return -ENOMEM; + + init_uobj(&obj->uevent.uobject, cmd.user_handle, file->ucontext, &qp_lock_key); + down_write(&obj->uevent.uobject.mutex); + + srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? + idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; + xrcd = cmd.qp_type == IB_QPT_XRC ? + idr_read_xrcd(cmd.srq_handle, file->ucontext, &xrcd_uobj) : NULL; + pd = idr_read_pd(cmd.pd_handle, file->ucontext); + scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); + rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? + scq : idr_read_cq(cmd.recv_cq_handle, file->ucontext, 1); + + if (!pd || !scq || !rcq || (cmd.is_srq && !srq) || + (cmd.qp_type == IB_QPT_XRC && !xrcd)) { + ret = -EINVAL; + goto err_put; + } + + attr.event_handler = ib_uverbs_qp_event_handler; + attr.qp_context = file; + attr.send_cq = scq; + attr.recv_cq = rcq; + attr.srq = srq; + attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; + attr.qp_type = cmd.qp_type; + attr.xrc_domain = xrcd; + attr.create_flags = cmd.reserved; + + attr.cap.max_send_wr = cmd.max_send_wr; + attr.cap.max_recv_wr = cmd.max_recv_wr; + attr.cap.max_send_sge = cmd.max_send_sge; + attr.cap.max_recv_sge = cmd.max_recv_sge; + attr.cap.max_inline_data = cmd.max_inline_data; + + obj->uevent.events_reported = 0; + INIT_LIST_HEAD(&obj->uevent.event_list); + INIT_LIST_HEAD(&obj->mcast_list); + + qp = pd->device->create_qp(pd, &attr, &udata); + if (IS_ERR(qp)) { + ret = PTR_ERR(qp); + goto err_put; + } + + qp->device = pd->device; + qp->pd = pd; + qp->send_cq = attr.send_cq; + qp->recv_cq = attr.recv_cq; + qp->srq = attr.srq; + qp->uobject = &obj->uevent.uobject; + qp->event_handler = attr.event_handler; + qp->qp_context = attr.qp_context; + qp->qp_type = attr.qp_type; + qp->xrcd = attr.xrc_domain; + atomic_inc(&pd->usecnt); + atomic_inc(&attr.send_cq->usecnt); + atomic_inc(&attr.recv_cq->usecnt); + if (attr.srq) + atomic_inc(&attr.srq->usecnt); + else if (attr.xrc_domain) + atomic_inc(&attr.xrc_domain->usecnt); + + obj->uevent.uobject.object = qp; + ret = idr_add_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); + if (ret) + goto err_destroy; + + memset(&resp, 0, sizeof resp); + resp.qpn = qp->qp_num; + resp.qp_handle = obj->uevent.uobject.id; + resp.max_recv_sge = attr.cap.max_recv_sge; + resp.max_send_sge = attr.cap.max_send_sge; + resp.max_recv_wr = attr.cap.max_recv_wr; + resp.max_send_wr = attr.cap.max_send_wr; + resp.max_inline_data = attr.cap.max_inline_data; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + ret = -EFAULT; + goto err_copy; + } + + put_pd_read(pd); + put_cq_read(scq); + if (rcq != scq) + put_cq_read(rcq); + if (srq) + put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd_uobj); + + mutex_lock(&file->mutex); + list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); + mutex_unlock(&file->mutex); + + obj->uevent.uobject.live = 1; + + up_write(&obj->uevent.uobject.mutex); + + return in_len; + +err_copy: + idr_remove_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); + +err_destroy: + ib_destroy_qp(qp); + +err_put: + if (pd) + put_pd_read(pd); + if (scq) + put_cq_read(scq); + if (rcq && rcq != scq) + put_cq_read(rcq); + if (srq) + put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd_uobj); + + put_uobj_write(&obj->uevent.uobject); + return ret; +} + + ssize_t ib_uverbs_query_qp(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 0d3f770..67719be 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -515,14 +515,8 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, if (err) goto err_mtt; } - } else { - qp->sq_no_prefetch = 0; - - if (init_attr->create_flags & IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) - qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; - if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) - qp->flags |= MLX4_IB_QP_LSO; + } else { err = set_kernel_sq_size(dev, &init_attr->cap, init_attr->qp_type, qp); if (err) @@ -705,8 +699,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)) return ERR_PTR(-EINVAL); - if (init_attr->create_flags && - (pd->uobject || init_attr->qp_type != IB_QPT_UD)) + if (init_attr->create_flags && init_attr->qp_type != IB_QPT_UD) return ERR_PTR(-EINVAL); switch (init_attr->qp_type) { @@ -722,11 +715,19 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, return ERR_PTR(-ENOMEM); err = create_qp_common(dev, pd, init_attr, udata, 0, qp); + if (err) { kfree(qp); return ERR_PTR(err); } + if (init_attr->create_flags & + IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) + qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; + + if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) + qp->flags |= MLX4_IB_QP_LSO; + if (init_attr->qp_type == IB_QPT_XRC) qp->xrcdn = to_mxrcd(init_attr->xrc_domain)->xrcdn; else From arlin.r.davis at intel.com Sun Jul 20 15:17:25 2008 From: arlin.r.davis at intel.com (Arlin Davis) Date: Sun, 20 Jul 2008 15:17:25 -0700 Subject: [ofa-general] [PATCH 1/4] Add IB UD extensions to v2 uDAPL package. Message-ID: <000001c8eab6$67129e40$67bd020a@amr.corp.intel.com> New definitions for IB unreliable datagram extension Extend EP dat_service_type, with DAT_IB_SERVICE_TYPE_UD Add IB extension call dat_ib_post_send_ud(). Add address handle definition for UD calls. Add IB event definitions to provide remote AH via connect and connect requests Roll IB extension version to 2.0.2 Signed-off by: Arlin Davis ardavis at ichips.intel.com --- dat/include/dat2/dat.h | 4 ++ dat/include/dat2/dat_ib_extensions.h | 67 +++++++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-) diff --git a/dat/include/dat2/dat.h b/dat/include/dat2/dat.h index d4e4cea..cf0b7ec 100755 --- a/dat/include/dat2/dat.h +++ b/dat/include/dat2/dat.h @@ -464,6 +464,10 @@ typedef DAT_UINT64 DAT_IA_ATTR_MASK; typedef enum dat_service_type { DAT_SERVICE_TYPE_RC /* reliable connections */ +#ifdef DAT_EXTENSIONS + ,DAT_SERVICE_TYPE_EXTENSION_BASE /* To be used by DAT extensions + as a starting point of extension services */ +#endif /* DAT_EXTENSIONS */ } DAT_SERVICE_TYPE; typedef struct dat_ep_attr diff --git a/dat/include/dat2/dat_ib_extensions.h b/dat/include/dat2/dat_ib_extensions.h index 0a9d3f5..27af51e 100755 --- a/dat/include/dat2/dat_ib_extensions.h +++ b/dat/include/dat2/dat_ib_extensions.h @@ -58,18 +58,24 @@ * DAT_NAMED_ATTR name == extended operations and version, * version_value = version number of extension API */ -#define DAT_IB_EXTENSION_VERSION 201 /* 2.0.1 */ +#define DAT_IB_EXTENSION_VERSION 202 /* 2.0.2 */ #define DAT_IB_ATTR_FETCH_AND_ADD "DAT_IB_FETCH_AND_ADD" #define DAT_IB_ATTR_CMP_AND_SWAP "DAT_IB_CMP_AND_SWAP" #define DAT_IB_ATTR_IMMED_DATA "DAT_IB_IMMED_DATA" +#define DAT_IB_ATTR_UD "DAT_IB_UD" /* * Definition for extended EVENT numbers, DAT_IB_EXTENSION_BASE_RANGE * is used by these extensions as a starting point for extended event numbers + * + * DAT_IB_DTO_EVENT - All extended data transfers - req/recv evd + * DAT_IB_AH_EVENT - address handle resolution - connect evd */ typedef enum dat_ib_event_number { DAT_IB_DTO_EVENT = DAT_IB_EXTENSION_RANGE_BASE, + DAT_IB_UD_CONNECTION_REQUEST_EVENT, + DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED } DAT_IB_EVENT_NUMBER; @@ -81,6 +87,7 @@ typedef enum dat_ib_op DAT_IB_FETCH_AND_ADD_OP, DAT_IB_CMP_AND_SWAP_OP, DAT_IB_RDMA_WRITE_IMMED_OP, + DAT_IB_UD_SEND_OP } DAT_IB_OP; @@ -98,6 +105,11 @@ typedef enum dat_ib_ext_type DAT_IB_CMP_AND_SWAP, // 1 DAT_IB_RDMA_WRITE_IMMED, // 2 DAT_IB_RDMA_WRITE_IMMED_DATA, // 3 + DAT_IB_RECV_IMMED_DATA, // 4 + DAT_IB_UD_CONNECT_REQUEST, // 5 + DAT_IB_UD_REMOTE_AH, // 6 + DAT_IB_UD_SEND, // 7 + DAT_IB_UD_RECV // 8 } DAT_IB_EXT_TYPE; @@ -133,6 +145,10 @@ typedef enum dat_ib_dtos DAT_IB_DTO_RECV_IMMED, DAT_IB_DTO_FETCH_ADD, DAT_IB_DTO_CMP_SWAP, + DAT_IB_DTO_RECV_MSG_IMMED, + DAT_IB_DTO_SEND_UD, + DAT_IB_DTO_RECV_UD, + DAT_IB_DTO_RECV_UD_IMMED, } DAT_IB_DTOS; @@ -155,7 +171,7 @@ typedef enum dat_ib_handle_type */ typedef enum dat_ib_evd_extension_flags { - DAT_IB_EVD_EXTENSION_FLAG = DAT_EVD_EXTENSION_BASE + DAT_IB_EVD_EXTENSION_FLAG = DAT_EVD_EXTENSION_BASE, } DAT_IB_EVD_EXTENSION_FLAGS; @@ -170,6 +186,16 @@ typedef enum dat_ib_mem_priv_flags } DAT_IB_MEM_PRIV_FLAGS; +/* + * Definition for IB address handle, unreliable datagram. + */ +typedef struct dat_ib_addr_handle +{ + struct ibv_ah *ah; + DAT_UINT32 qpn; + DAT_SOCK_ADDR6 ia_addr; + +} DAT_IB_ADDR_HANDLE; /* * Definitions for extended event data: @@ -194,6 +220,9 @@ typedef struct dat_ib_immed_data * specifies extension data values. * NOTE: DAT_EXTENSION_EVENT_DATA cannot exceed 64 bytes as defined by * "DAT_UINT64 extension_data[8]" in DAT_EVENT (dat.h) + * + * Provide UD address handles via extended connect establishment. + * ia_addr provided with extended conn events for reference. */ typedef struct dat_ib_extension_event_data { @@ -202,9 +231,22 @@ typedef struct dat_ib_extension_event_data union { DAT_IB_IMMED_DATA immed; } val; + DAT_IB_ADDR_HANDLE remote_ah; } DAT_IB_EXTENSION_EVENT_DATA; +/* + * Definitions for additional extension handle types beyond + * standard DAT handle. New Bit definitions MUST start at + * DAT_HANDLE_TYPE_EXTENSION_BASE + */ +typedef enum dat_ib_service_type +{ + DAT_IB_SERVICE_TYPE_UD = DAT_SERVICE_TYPE_EXTENSION_BASE, + +} DAT_IB_SERVICE_TYPE; + + /* Extended RETURN and EVENT STATUS string helper functions */ /* DAT_EXT_RETURN error to string */ @@ -306,5 +348,26 @@ dat_strerror_ext_status ( (idata), \ (flgs)) +/* + * Unreliable datagram: msg send + * + * This asynchronous call is modeled after the InfiniBand UD message send + * Event completion for the request completes as an + * DAT_EXTENSION with extension type set to DAT_DTO_EXTENSION_UD_SEND. + * Event completion on the remote endpoint completes as receive DTO operation + * type of DAT_EXTENSION with operation set to DAT_DTO_EXTENSION_UD_RECV. + * + * Other extension flags: + * n/a + */ +#define dat_ib_post_send_ud(ep, segments, lbuf, ah_ptr, cookie, flgs) \ + dat_extension_op( ep, \ + DAT_IB_UD_SEND_OP, \ + (segments), \ + (lbuf), \ + (ah_ptr), \ + (cookie), \ + (flgs)) + #endif /* _DAT_IB_EXTENSIONS_H_ */ -- 1.5.2.5 From arlin.r.davis at intel.com Sun Jul 20 15:18:18 2008 From: arlin.r.davis at intel.com (Arlin Davis) Date: Sun, 20 Jul 2008 15:18:18 -0700 Subject: [ofa-general] [PATCH 2/4] uDAPL v2 - IB UD extension - dapl: add support for UD extensions in common code Message-ID: <000101c8eab6$84d44370$67bd020a@amr.corp.intel.com> allow EP create for extended service types. extend connection event types to include UD AH resolution/exchange. add new extended connect and connect request upcalls for providers. - dapls_evd_post_cr_event_ext - dapls_evd_post_connection_event_ext Signed-off by: Arlin Davis ardavis at ichips.intel.com --- dapl/common/dapl_ep_create.c | 6 +- dapl/common/dapl_evd_util.c | 169 ++++++++++++++++++++++++++++++++++++++++++ dapl/common/dapl_evd_util.h | 20 +++++ 3 files changed, 193 insertions(+), 2 deletions(-) diff --git a/dapl/common/dapl_ep_create.c b/dapl/common/dapl_ep_create.c index 9e6bd9c..ff233b0 100644 --- a/dapl/common/dapl_ep_create.c +++ b/dapl/common/dapl_ep_create.c @@ -160,8 +160,10 @@ dapl_ep_create ( * max_*_dtos must 0 as the user will not be able to post dto ops on * the respective queue. */ - if (ep_attr != NULL && - (ep_attr->service_type != DAT_SERVICE_TYPE_RC || + if (ep_attr != NULL && ( +#ifndef DAT_EXTENSIONS + ep_attr->service_type != DAT_SERVICE_TYPE_RC || +#endif (recv_evd_handle == DAT_HANDLE_NULL && ep_attr->max_recv_dtos != 0) || (recv_evd_handle != DAT_HANDLE_NULL && ep_attr->max_recv_dtos == 0) || (request_evd_handle == DAT_HANDLE_NULL && ep_attr->max_request_dtos != 0) || diff --git a/dapl/common/dapl_evd_util.c b/dapl/common/dapl_evd_util.c index 63fee81..9270750 100644 --- a/dapl/common/dapl_evd_util.c +++ b/dapl/common/dapl_evd_util.c @@ -44,6 +44,9 @@ #include "dapl_adapter_util.h" #include "dapl_cookie.h" #include "dapl.h" +#include "dapl_cr_util.h" +#include "dapl_sp_util.h" +#include "dapl_ep_util.h" #include #include @@ -84,6 +87,8 @@ char *dapl_event_str( IN DAT_EVENT_NUMBER event_num ) #ifdef DAT_EXTENSIONS {"DAT_EXTENSION_EVENT", DAT_EXTENSION_EVENT}, {"DAT_IB_EXTENSION_RANGE_BASE", DAT_IB_EXTENSION_RANGE_BASE}, + {"DAT_IB_UD_CONNECTION_REQUEST_EVENT", DAT_IB_EXTENSION_RANGE_BASE+1}, + {"DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED", DAT_IB_EXTENSION_RANGE_BASE+2}, {"DAT_IW_EXTENSION_RANGE_BASE", DAT_IW_EXTENSION_RANGE_BASE}, #endif /* DAT_EXTENSIONS */ {NULL,0}, @@ -1048,6 +1053,170 @@ dapls_evd_post_generic_event ( return DAT_SUCCESS; } +#ifdef DAT_EXTENSIONS +DAT_RETURN +dapls_evd_post_cr_event_ext ( + IN DAPL_SP *sp_ptr, + IN DAT_EVENT_NUMBER event_number, + IN dp_ib_cm_handle_t ib_cm_handle, + IN DAT_COUNT p_size, + IN DAT_PVOID p_data, + IN DAT_PVOID ext_data) +{ + DAPL_CR *cr_ptr; + DAPL_EP *ep_ptr; + DAT_EVENT *event_ptr; + DAT_SP_HANDLE sp_handle; + + dapl_os_lock (&sp_ptr->header.lock); + if (sp_ptr->listening == DAT_FALSE) { + dapl_os_unlock (&sp_ptr->header.lock); + dapl_dbg_log(DAPL_DBG_TYPE_CM, + "---> post_cr_event_ext: conn event on down SP\n"); + (void)dapls_ib_reject_connection(ib_cm_handle, + DAT_CONNECTION_EVENT_UNREACHABLE, + 0, NULL); + return DAT_CONN_QUAL_UNAVAILABLE; + } + + /* + * RSP connections only allow a single connection. Close + * it down NOW so we reject any further connections. + */ + if (sp_ptr->header.handle_type == DAT_HANDLE_TYPE_RSP) + sp_ptr->listening = DAT_FALSE; + + dapl_os_unlock (&sp_ptr->header.lock); + + /* allocate new connect request */ + cr_ptr = dapls_cr_alloc(sp_ptr->header.owner_ia); + if (cr_ptr == NULL) + return DAT_INSUFFICIENT_RESOURCES; + + /* Set up the CR */ + cr_ptr->sp_ptr = sp_ptr; /* maintain sp_ptr in case of reject */ + cr_ptr->param.remote_port_qual = 0; + cr_ptr->ib_cm_handle = ib_cm_handle; + cr_ptr->param.remote_ia_address_ptr = (DAT_IA_ADDRESS_PTR)&cr_ptr->remote_ia_address; + + /* + * Copy the remote address and private data out of the private_data + */ + cr_ptr->param.private_data = cr_ptr->private_data; + cr_ptr->param.private_data_size = p_size; + if (p_size) + dapl_os_memcpy(cr_ptr->private_data, p_data, p_size); + + /* EP will be NULL unless RSP service point */ + ep_ptr = (DAPL_EP *)sp_ptr->ep_handle; + + if (sp_ptr->psp_flags == DAT_PSP_PROVIDER_FLAG) + { + DAPL_IA *ia_ptr; + /* + * Never true for RSP connections + * + * Create an EP for the user. If we can't allocate an + * EP we are out of resources and need to tell the + * requestor that we cant help them. + */ + ia_ptr = sp_ptr->header.owner_ia; + ep_ptr = dapl_ep_alloc(ia_ptr, NULL); + if (ep_ptr == NULL) { + dapls_cr_free (cr_ptr); + /* Invoking function will call dapls_ib_cm_reject() */ + return DAT_INSUFFICIENT_RESOURCES; + } + ep_ptr->param.ia_handle = ia_ptr; + ep_ptr->param.local_ia_address_ptr = + (DAT_IA_ADDRESS_PTR)&ia_ptr->hca_ptr->hca_address; + + /* Link the EP onto the IA */ + dapl_ia_link_ep(ia_ptr, ep_ptr); + } + + cr_ptr->param.local_ep_handle = ep_ptr; + + if (ep_ptr != NULL) { + /* Assign valid EP fields: RSP and PSP_PROVIDER_FLAG only */ + if ( sp_ptr->psp_flags == DAT_PSP_PROVIDER_FLAG ) { + ep_ptr->param.ep_state = + DAT_EP_STATE_TENTATIVE_CONNECTION_PENDING; + } else { + /* RSP */ + dapl_os_assert (sp_ptr->header.handle_type == DAT_HANDLE_TYPE_RSP); + ep_ptr->param.ep_state = + DAT_EP_STATE_PASSIVE_CONNECTION_PENDING; + } + ep_ptr->cm_handle = ib_cm_handle; + } + + /* link the CR onto the SP so we can pick it up later */ + dapl_sp_link_cr(sp_ptr, cr_ptr); + + /* assign sp_ptr to union to avoid typecast errors from some compilers */ + sp_handle.psp_handle = (DAT_PSP_HANDLE)sp_ptr; + + /* Post the event. */ + + /* + * Note event lock may be held on successful return + * to be released by dapli_evd_post_event(), if provider side locking + * is needed. + */ + event_ptr = dapli_evd_get_and_init_event(sp_ptr->evd_handle, + event_number); + if (event_ptr == NULL) + return DAT_ERROR(DAT_INSUFFICIENT_RESOURCES, DAT_RESOURCE_MEMORY); + + event_ptr->event_data.cr_arrival_event_data.sp_handle = sp_handle; + event_ptr->event_data.cr_arrival_event_data.local_ia_address_ptr = + (DAT_IA_ADDRESS_PTR)&sp_ptr->header.owner_ia->hca_ptr->hca_address; + event_ptr->event_data.cr_arrival_event_data.conn_qual = sp_ptr->conn_qual; + event_ptr->event_data.cr_arrival_event_data.cr_handle = (DAT_HANDLE)cr_ptr; + + dapl_os_memcpy(&event_ptr->event_extension_data[0], ext_data, 64); + + dapli_evd_post_event(sp_ptr->evd_handle, event_ptr); + + return DAT_SUCCESS; +} + + +DAT_RETURN +dapls_evd_post_connection_event_ext ( + IN DAPL_EVD *evd_ptr, + IN DAT_EVENT_NUMBER event_number, + IN DAT_EP_HANDLE ep_handle, + IN DAT_COUNT private_data_size, + IN DAT_PVOID private_data, + IN DAT_PVOID ext_data) +{ + DAT_EVENT *event_ptr; + event_ptr = dapli_evd_get_and_init_event (evd_ptr, event_number); + /* + * Note event lock may be held on successful return + * to be released by dapli_evd_post_event(), if provider side locking + * is needed. + */ + if (event_ptr == NULL) + return DAT_ERROR (DAT_INSUFFICIENT_RESOURCES, DAT_RESOURCE_MEMORY); + + event_ptr->event_data.connect_event_data.ep_handle = ep_handle; + event_ptr->event_data.connect_event_data.private_data_size + = private_data_size; + event_ptr->event_data.connect_event_data.private_data = private_data; + + dapl_os_memcpy(&event_ptr->event_extension_data[0], ext_data, 64); + + dapli_evd_post_event (evd_ptr, event_ptr); + + return DAT_SUCCESS; +} +#endif + + + /* * dapli_evd_cqe_to_event * diff --git a/dapl/common/dapl_evd_util.h b/dapl/common/dapl_evd_util.h index 563bd67..2304435 100644 --- a/dapl/common/dapl_evd_util.h +++ b/dapl/common/dapl_evd_util.h @@ -109,6 +109,26 @@ dapls_evd_post_generic_event ( IN DAT_EVENT_NUMBER event_number, IN DAT_EVENT_DATA *data); +#ifdef DAT_EXTENSIONS +DAT_RETURN +dapls_evd_post_cr_event_ext ( + IN DAPL_SP *sp_ptr, + IN DAT_EVENT_NUMBER event_number, + IN dp_ib_cm_handle_t ib_cm_handle, + IN DAT_COUNT p_size, + IN DAT_PVOID p_data, + IN DAT_PVOID ext_data); + +DAT_RETURN +dapls_evd_post_connection_event_ext ( + IN DAPL_EVD *evd_ptr, + IN DAT_EVENT_NUMBER event_number, + IN DAT_EP_HANDLE ep_handle, + IN DAT_COUNT private_data_size, + IN DAT_PVOID private_data, + IN DAT_PVOID ext_data); +#endif + /************************************* * dapl internal callbacks functions * *************************************/ -- 1.5.2.5 From arlin.r.davis at intel.com Sun Jul 20 15:19:11 2008 From: arlin.r.davis at intel.com (Arlin Davis) Date: Sun, 20 Jul 2008 15:19:11 -0700 Subject: [ofa-general] [PATCH 3/4] uDAPL v2 - IB UD extension - dapl scm: add support for UD extensions in socket cm provider Message-ID: <000201c8eab6$a4650350$67bd020a@amr.corp.intel.com> add qp_type in connection information exchange add new post_send_ud call changes to connection manager to support qp types beyond RC. changes to connection events to use new extended event calls. exchange address handle information during connection phase. changes to modify_qp to handle both RC and UD types. Signed-off by: Arlin Davis ardavis at ichips.intel.com --- dapl/openib_scm/dapl_ib_cm.c | 157 ++++++++++++++++++++++++++-------- dapl/openib_scm/dapl_ib_dto.h | 62 ++++++++++++-- dapl/openib_scm/dapl_ib_extensions.c | 61 ++++++++++++-- dapl/openib_scm/dapl_ib_qp.c | 104 +++++++++++++++------- dapl/openib_scm/dapl_ib_util.c | 14 ++- dapl/openib_scm/dapl_ib_util.h | 22 ++++- 6 files changed, 324 insertions(+), 96 deletions(-) diff --git a/dapl/openib_scm/dapl_ib_cm.c b/dapl/openib_scm/dapl_ib_cm.c index b87c060..e712f9d 100644 --- a/dapl/openib_scm/dapl_ib_cm.c +++ b/dapl/openib_scm/dapl_ib_cm.c @@ -243,6 +243,7 @@ dapli_socket_connect(DAPL_EP *ep_ptr, /* Send QP info, IA address, and private data */ cm_ptr->dst.qpn = htonl(ep_ptr->qp_handle->qp_num); + cm_ptr->dst.qp_type = htons(ep_ptr->qp_handle->qp_type); cm_ptr->dst.port = htons(ia_ptr->hca_ptr->port_num); cm_ptr->dst.lid = htons(dapli_get_lid(ia_ptr->hca_ptr->ib_hca_handle, @@ -266,7 +267,10 @@ dapli_socket_connect(DAPL_EP *ep_ptr, iovec[1].iov_len = p_size; } - dapl_dbg_log(DAPL_DBG_TYPE_EP," socket connected, write QP and private data\n"); + dapl_dbg_log(DAPL_DBG_TYPE_EP, + " socket connected, write QP (%d), private data (%d)\n", + sizeof(ib_qp_cm_t),p_size); + len = writev(cm_ptr->socket, iovec, (p_size ? 2:1)); if (len != (p_size + sizeof(ib_qp_cm_t))) { dapl_dbg_log(DAPL_DBG_TYPE_ERR, @@ -319,7 +323,7 @@ dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr) if (len != sizeof(ib_qp_cm_t) || ntohs(cm_ptr->dst.ver) != DSCM_VER) { dapl_dbg_log(DAPL_DBG_TYPE_ERR, " connect_rtu read: ERR %s, rcnt=%d, ver=%d\n", - strerror(errno), len, cm_ptr->dst.ver); + strerror(errno), len, ntohs(cm_ptr->dst.ver)); goto bail; } /* check for consumer reject */ @@ -335,6 +339,7 @@ dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr) cm_ptr->dst.port = ntohs(cm_ptr->dst.port); cm_ptr->dst.lid = ntohs(cm_ptr->dst.lid); cm_ptr->dst.qpn = ntohl(cm_ptr->dst.qpn); + cm_ptr->dst.qp_type = ntohs(cm_ptr->dst.qp_type); cm_ptr->dst.p_size = ntohl(cm_ptr->dst.p_size); /* save remote address information */ @@ -343,10 +348,13 @@ dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr) sizeof(ep_ptr->remote_ia_address)); dapl_dbg_log(DAPL_DBG_TYPE_EP, - " connect_rtu: DST %s port=0x%x lid=0x%x, qpn=0x%x, psize=%d\n", - inet_ntoa(((struct sockaddr_in *)&cm_ptr->dst.ia_address)->sin_addr), + " connect_rtu: DST %s port=0x%x lid=0x%x," + " qpn=0x%x, qp_type=%d, psize=%d\n", + inet_ntoa(((struct sockaddr_in *) + &cm_ptr->dst.ia_address)->sin_addr), cm_ptr->dst.port, cm_ptr->dst.lid, - cm_ptr->dst.qpn, cm_ptr->dst.p_size); + cm_ptr->dst.qpn, cm_ptr->dst.qp_type, + cm_ptr->dst.p_size); /* validate private data size before reading */ if (cm_ptr->dst.p_size > IB_MAX_REP_PDATA_SIZE) { @@ -357,7 +365,8 @@ dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr) } /* read private data into cm_handle if any present */ - dapl_dbg_log(DAPL_DBG_TYPE_EP," socket connected, read private data\n"); + dapl_dbg_log(DAPL_DBG_TYPE_EP," socket connected, read pdata\n"); + if (cm_ptr->dst.p_size) { iovec[0].iov_base = cm_ptr->p_data; iovec[0].iov_len = cm_ptr->dst.p_size; @@ -372,11 +381,11 @@ dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr) /* modify QP to RTR and then to RTS with remote info */ if (dapls_modify_qp_state(ep_ptr->qp_handle, - IBV_QPS_RTR, &cm_ptr->dst) != DAT_SUCCESS) + IBV_QPS_RTR, cm_ptr) != DAT_SUCCESS) goto bail; if (dapls_modify_qp_state(ep_ptr->qp_handle, - IBV_QPS_RTS, &cm_ptr->dst) != DAT_SUCCESS) + IBV_QPS_RTS, cm_ptr) != DAT_SUCCESS) goto bail; ep_ptr->qp_state = IB_QP_STATE_RTS; @@ -390,10 +399,34 @@ dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr) ep_ptr->cm_handle = cm_ptr; cm_ptr->state = SCM_CONNECTED; dapl_dbg_log(DAPL_DBG_TYPE_EP," ACTIVE: connected!\n"); + +#ifdef DAT_EXTENSIONS + if (cm_ptr->dst.qp_type == IBV_QPT_UD) { + DAT_IB_EXTENSION_EVENT_DATA xevent; + + /* post EVENT, modify_qp created ah */ + xevent.status = 0; + xevent.type = DAT_IB_UD_REMOTE_AH; + xevent.remote_ah.ah = cm_ptr->ah; + xevent.remote_ah.qpn = cm_ptr->dst.qpn; + dapl_os_memcpy( &xevent.remote_ah.ia_addr, + &cm_ptr->dst.ia_address, + sizeof(cm_ptr->dst.ia_address)); + + dapls_evd_post_connection_event_ext( + (DAPL_EVD*)ep_ptr->param.connect_evd_handle, + DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED, + (DAT_EP_HANDLE)ep_ptr, + (DAT_COUNT)cm_ptr->dst.p_size, + (DAT_PVOID*)cm_ptr->p_data, + (DAT_PVOID*)&xevent); + } else +#endif dapl_evd_connection_callback(cm_ptr, IB_CME_CONNECTED, cm_ptr->p_data, ep_ptr); + return; bail: /* close socket, free cm structure and post error event */ @@ -515,7 +548,7 @@ dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr) ntohs(acm_ptr->dst.ver) != DSCM_VER) { dapl_dbg_log(DAPL_DBG_TYPE_ERR, " accept read: ERR %s, rcnt=%d, ver=%d\n", - strerror(errno), len, acm_ptr->dst.ver); + strerror(errno), len, ntohs(acm_ptr->dst.ver)); dat_status = DAT_INTERNAL_ERROR; goto bail; } @@ -524,11 +557,13 @@ dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr) acm_ptr->dst.port = ntohs(acm_ptr->dst.port); acm_ptr->dst.lid = ntohs(acm_ptr->dst.lid); acm_ptr->dst.qpn = ntohl(acm_ptr->dst.qpn); + acm_ptr->dst.qp_type = ntohs(acm_ptr->dst.qp_type); acm_ptr->dst.p_size = ntohl(acm_ptr->dst.p_size); dapl_dbg_log(DAPL_DBG_TYPE_EP, - " accept: DST %s port=0x%x lid=0x%x, qpn=0x%x, psize=%d\n", - inet_ntoa(((struct sockaddr_in *)&acm_ptr->dst.ia_address)->sin_addr), + " accept: DST %s port=0x%x lid=0x%x, qpn=0x%x, psz=%d\n", + inet_ntoa(((struct sockaddr_in *) + &acm_ptr->dst.ia_address)->sin_addr), acm_ptr->dst.port, acm_ptr->dst.lid, acm_ptr->dst.qpn, acm_ptr->dst.p_size); @@ -559,7 +594,24 @@ dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr) } acm_ptr->state = SCM_ACCEPTING; + +#ifdef DAT_EXTENSIONS + if (acm_ptr->dst.qp_type == IBV_QPT_UD) { + DAT_IB_EXTENSION_EVENT_DATA xevent; + /* post EVENT, modify_qp created ah */ + xevent.status = 0; + xevent.type = DAT_IB_UD_CONNECT_REQUEST; + + dapls_evd_post_cr_event_ext( + acm_ptr->sp, + DAT_IB_UD_CONNECTION_REQUEST_EVENT, + acm_ptr, + (DAT_COUNT)acm_ptr->dst.p_size, + (DAT_PVOID*)acm_ptr->p_data, + (DAT_PVOID*)&xevent); + } else +#endif /* trigger CR event and return SUCCESS */ dapls_cr_callback(acm_ptr, IB_CME_CONNECTION_REQUEST_PENDING, @@ -584,6 +636,7 @@ dapli_socket_accept_usr(DAPL_EP *ep_ptr, { DAPL_IA *ia_ptr = ep_ptr->header.owner_ia; dp_ib_cm_handle_t cm_ptr = cr_ptr->ib_cm_handle; + ib_qp_cm_t local; struct iovec iovec[2]; int len; @@ -596,44 +649,59 @@ dapli_socket_accept_usr(DAPL_EP *ep_ptr, dapl_dbg_log(DAPL_DBG_TYPE_EP, " accept_usr: remote port=0x%x lid=0x%x" - " qpn=0x%x psize=%d\n", + " qpn=0x%x qp_type %d, psize=%d\n", cm_ptr->dst.port, cm_ptr->dst.lid, - cm_ptr->dst.qpn, cm_ptr->dst.p_size); + cm_ptr->dst.qpn, cm_ptr->dst.qp_type, + cm_ptr->dst.p_size); + +#ifdef DAT_EXTENSIONS + if (cm_ptr->dst.qp_type == IBV_QPT_UD && + ep_ptr->qp_handle->qp_type != IBV_QPT_UD) { + dapl_dbg_log(DAPL_DBG_TYPE_ERR, + " accept_rtu: ERR remote QP is UD," + ", but local QP is not\n"); + return (DAT_INVALID_HANDLE | DAT_INVALID_HANDLE_EP); + + } +#endif /* modify QP to RTR and then to RTS with remote info already read */ if (dapls_modify_qp_state(ep_ptr->qp_handle, - IBV_QPS_RTR, &cm_ptr->dst) != DAT_SUCCESS) + IBV_QPS_RTR, cm_ptr) != DAT_SUCCESS) goto bail; if (dapls_modify_qp_state(ep_ptr->qp_handle, - IBV_QPS_RTS, &cm_ptr->dst) != DAT_SUCCESS) + IBV_QPS_RTS, cm_ptr) != DAT_SUCCESS) goto bail; ep_ptr->qp_state = IB_QP_STATE_RTS; - /* save remote address information */ + /* save remote address information, for qp_query */ dapl_os_memcpy( &ep_ptr->remote_ia_address, &cm_ptr->dst.ia_address, sizeof(ep_ptr->remote_ia_address)); - /* send our QP info, IA address, and private data */ - cm_ptr->dst.qpn = htonl(ep_ptr->qp_handle->qp_num); - cm_ptr->dst.port = htons(ia_ptr->hca_ptr->port_num); - cm_ptr->dst.lid = htons(dapli_get_lid(ia_ptr->hca_ptr->ib_hca_handle, + /* send our QP info, IA address, pdata. Don't overwrite dst data */ + local.ver = htons(DSCM_VER); + local.rej = 0; + local.qpn = htonl(ep_ptr->qp_handle->qp_num); + local.qp_type = htons(ep_ptr->qp_handle->qp_type); + local.port = htons(ia_ptr->hca_ptr->port_num); + local.lid = htons(dapli_get_lid(ia_ptr->hca_ptr->ib_hca_handle, (uint8_t)ia_ptr->hca_ptr->port_num)); - if (cm_ptr->dst.lid == 0xffff) + if (local.lid == 0xffff) goto bail; /* in network order */ if (ibv_query_gid(ia_ptr->hca_ptr->ib_hca_handle, (uint8_t)ia_ptr->hca_ptr->port_num, 0, - &cm_ptr->dst.gid)) + &local.gid)) goto bail; - cm_ptr->dst.ia_address = ia_ptr->hca_ptr->hca_address; - cm_ptr->dst.p_size = htonl(p_size); - iovec[0].iov_base = &cm_ptr->dst; + local.ia_address = ia_ptr->hca_ptr->hca_address; + local.p_size = htonl(p_size); + iovec[0].iov_base = &local; iovec[0].iov_len = sizeof(ib_qp_cm_t); if (p_size) { iovec[1].iov_base = p_data; @@ -648,26 +716,22 @@ dapli_socket_accept_usr(DAPL_EP *ep_ptr, } dapl_dbg_log(DAPL_DBG_TYPE_CM, " accept_usr: local port=0x%x lid=0x%x" - " qpn=0x%x psize=%d\n", - ntohs(cm_ptr->dst.port), ntohs(cm_ptr->dst.lid), - ntohl(cm_ptr->dst.qpn), ntohl(cm_ptr->dst.p_size)); + " qpn=0x%x qp_type=%d psize=%d\n", + ntohs(local.port), ntohs(local.lid), + ntohl(local.qpn), ntohs(local.qp_type), + ntohl(local.p_size)); dapl_dbg_log(DAPL_DBG_TYPE_CM, " accept_usr SRC GID subnet %016llx id %016llx\n", (unsigned long long) - cpu_to_be64(cm_ptr->dst.gid.global.subnet_prefix), + cpu_to_be64(local.gid.global.subnet_prefix), (unsigned long long) - cpu_to_be64(cm_ptr->dst.gid.global.interface_id)); + cpu_to_be64(local.gid.global.interface_id)); /* save state and reference to EP, queue for RTU data */ cm_ptr->ep = ep_ptr; cm_ptr->hca = ia_ptr->hca_ptr; cm_ptr->state = SCM_ACCEPTED; - /* restore remote address information for query */ - dapl_os_memcpy( &cm_ptr->dst.ia_address, - &ep_ptr->remote_ia_address, - sizeof(cm_ptr->dst.ia_address)); - dapl_dbg_log( DAPL_DBG_TYPE_EP," PASSIVE: accepted!\n" ); dapli_cm_queue(cm_ptr); return DAT_SUCCESS; @@ -701,6 +765,29 @@ dapli_socket_accept_rtu(dp_ib_cm_handle_t cm_ptr) /* final data exchange if remote QP state is good to go */ dapl_dbg_log( DAPL_DBG_TYPE_EP," PASSIVE: connected!\n" ); + +#ifdef DAT_EXTENSIONS + if (cm_ptr->dst.qp_type == IBV_QPT_UD) { + DAT_IB_EXTENSION_EVENT_DATA xevent; + + /* post EVENT, modify_qp created ah */ + xevent.status = 0; + xevent.type = DAT_IB_UD_REMOTE_AH; + xevent.remote_ah.ah = cm_ptr->ah; + xevent.remote_ah.qpn = cm_ptr->dst.qpn; + dapl_os_memcpy( &xevent.remote_ah.ia_addr, + &cm_ptr->dst.ia_address, + sizeof(cm_ptr->dst.ia_address)); + + dapls_evd_post_connection_event_ext( + (DAPL_EVD*)cm_ptr->ep->param.connect_evd_handle, + DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED, + (DAT_EP_HANDLE)cm_ptr->ep, + (DAT_COUNT)cm_ptr->dst.p_size, + (DAT_PVOID*)cm_ptr->p_data, + (DAT_PVOID*)&xevent); + } else +#endif dapls_cr_callback(cm_ptr, IB_CME_CONNECTED, NULL, cm_ptr->sp); return; bail: diff --git a/dapl/openib_scm/dapl_ib_dto.h b/dapl/openib_scm/dapl_ib_dto.h index 4b06b72..b9826f5 100644 --- a/dapl/openib_scm/dapl_ib_dto.h +++ b/dapl/openib_scm/dapl_ib_dto.h @@ -58,6 +58,9 @@ STATIC _INLINE_ int dapls_cqe_opcode(ib_work_completion_t *cqe_p); +#define CQE_WR_TYPE_UD(id) \ + (((DAPL_COOKIE *)(uintptr_t)id)->ep->qp_handle->qp_type == IBV_QPT_UD) + /* * dapls_ib_post_recv * @@ -171,7 +174,11 @@ dapls_ib_post_send ( if (NULL == ds_array_p) return (DAT_INSUFFICIENT_RESOURCES); - + +#ifdef DAT_EXTENSIONS + if (ep_ptr->qp_handle->qp_type != IBV_QPT_RC) + return(DAT_ERROR(DAT_INVALID_HANDLE, DAT_INVALID_HANDLE_EP)); +#endif /* setup the work request */ wr.next = 0; wr.opcode = op_type; @@ -247,6 +254,11 @@ STATIC _INLINE_ DAT_DTOS dapls_cqe_dtos_opcode(ib_work_completion_t *cqe_p) switch (cqe_p->opcode) { case IBV_WC_SEND: +#ifdef DAT_EXTENSIONS + if (CQE_WR_TYPE_UD(cqe_p->wr_id)) + return (DAT_IB_DTO_SEND_UD); + else +#endif return (DAT_DTO_SEND); case IBV_WC_RDMA_READ: return (DAT_DTO_RDMA_READ); @@ -269,6 +281,13 @@ STATIC _INLINE_ DAT_DTOS dapls_cqe_dtos_opcode(ib_work_completion_t *cqe_p) return (DAT_DTO_RDMA_WRITE); #endif case IBV_WC_RECV: +#ifdef DAT_EXTENSIONS + if (CQE_WR_TYPE_UD(cqe_p->wr_id)) + return (DAT_IB_DTO_RECV_UD); + else if (cqe_p->wc_flags & IBV_WC_WITH_IMM) + return (DAT_IB_DTO_RECV_MSG_IMMED); + else +#endif return (DAT_DTO_RECEIVE); default: return (0xff); @@ -295,13 +314,14 @@ dapls_ib_post_ext_send ( IN DAT_UINT32 immed_data, IN DAT_UINT64 compare_add, IN DAT_UINT64 swap, - IN DAT_COMPLETION_FLAGS completion_flags) + IN DAT_COMPLETION_FLAGS completion_flags, + IN DAT_IB_ADDR_HANDLE *remote_ah) { dapl_dbg_log(DAPL_DBG_TYPE_EP, - " post_snd: ep %p op %d ck %p sgs", + " post_ext_snd: ep %p op %d ck %p sgs", "%d l_iov %p r_iov %p f %d\n", ep_ptr, op_type, cookie, segments, local_iov, - remote_iov, completion_flags); + remote_iov, completion_flags, remote_ah); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; ib_data_segment_t *ds_array_p, *ds_array_start_p; @@ -393,6 +413,21 @@ dapls_ib_post_ext_send ( wr.wr.atomic.remote_addr = remote_iov->virtual_address; wr.wr.atomic.rkey = remote_iov->rmr_context; break; + case OP_SEND_UD: + /* post must be on EP with service_type of UD */ + if (ep_ptr->qp_handle->qp_type != IBV_QPT_UD) + return(DAT_ERROR(DAT_INVALID_HANDLE, DAT_INVALID_HANDLE_EP)); + + dapl_dbg_log(DAPL_DBG_TYPE_EP, + " post_ext: OP_SEND_UD ah=%p" + " qp_num=0x%x\n", + remote_ah, remote_ah->qpn); + + wr.opcode = OP_SEND; + wr.wr.ud.ah = remote_ah->ah; + wr.wr.ud.remote_qpn = remote_ah->qpn; + wr.wr.ud.remote_qkey = SCM_UD_QKEY; + break; default: break; } @@ -431,12 +466,16 @@ dapls_ib_optional_prv_dat( return DAT_SUCCESS; } + /* map Work Completions to DAPL WR operations */ STATIC _INLINE_ int dapls_cqe_opcode(ib_work_completion_t *cqe_p) { switch (cqe_p->opcode) { case IBV_WC_SEND: - return (OP_SEND); + if (CQE_WR_TYPE_UD(cqe_p->wr_id)) + return(OP_SEND_UD); + else + return (OP_SEND); case IBV_WC_RDMA_WRITE: if (cqe_p->wc_flags & IBV_WC_WITH_IMM) return (OP_RDMA_WRITE_IMM); @@ -451,8 +490,10 @@ STATIC _INLINE_ int dapls_cqe_opcode(ib_work_completion_t *cqe_p) case IBV_WC_BIND_MW: return (OP_BIND_MW); case IBV_WC_RECV: - if (cqe_p->wc_flags & IBV_WC_WITH_IMM) - return (OP_RECEIVE_IMM); + if (CQE_WR_TYPE_UD(cqe_p->wr_id)) + return (OP_RECV_UD); + else if (cqe_p->wc_flags & IBV_WC_WITH_IMM) + return (OP_RECEIVE_MSG_IMM); else return (OP_RECEIVE); case IBV_WC_RECV_RDMA_WITH_IMM: @@ -481,10 +522,13 @@ STATIC _INLINE_ char * dapls_dto_op_str(int op) "OP_COMP_AND_SWAP", "OP_FETCH_AND_ADD", "OP_RECEIVE", - "OP_RECEIVE_IMM", + "OP_RECEIVE_MSG_IMM", + "OP_RECEIVE_RDMA_IMM", "OP_BIND_MW" + "OP_SEND_UD" + "OP_RECV_UD" }; - return ((op < 0 || op > 9) ? "Invalid CQE OP?" : optable[op]); + return ((op < 0 || op > 12) ? "Invalid CQE OP?" : optable[op]); } static _INLINE_ char * diff --git a/dapl/openib_scm/dapl_ib_extensions.c b/dapl/openib_scm/dapl_ib_extensions.c index 1402057..b88e853 100755 --- a/dapl/openib_scm/dapl_ib_extensions.c +++ b/dapl/openib_scm/dapl_ib_extensions.c @@ -54,7 +54,8 @@ dapli_post_ext( IN DAT_EP_HANDLE ep_handle, IN DAT_DTO_COOKIE user_cookie, IN const DAT_RMR_TRIPLET *remote_iov, IN int op_type, - IN DAT_COMPLETION_FLAGS flags ); + IN DAT_COMPLETION_FLAGS flags, + IN DAT_IB_ADDR_HANDLE *ah); /* @@ -81,6 +82,7 @@ dapl_extensions(IN DAT_HANDLE dat_handle, IN va_list args) { DAT_EP_HANDLE ep; + DAT_IB_ADDR_HANDLE *ah; DAT_LMR_TRIPLET *lmr_p; DAT_DTO_COOKIE cookie; const DAT_RMR_TRIPLET *rmr_p; @@ -113,7 +115,7 @@ dapl_extensions(IN DAT_HANDLE dat_handle, status = dapli_post_ext(ep, 0, 0, dat_uint32, segments, lmr_p, cookie, rmr_p, OP_RDMA_WRITE_IMM, - comp_flags ); + comp_flags, ah); break; case DAT_IB_CMP_AND_SWAP_OP: @@ -130,7 +132,7 @@ dapl_extensions(IN DAT_HANDLE dat_handle, status = dapli_post_ext(ep, dat_uint64a, dat_uint64b, 0, segments, lmr_p, cookie, rmr_p, - OP_COMP_AND_SWAP, comp_flags ); + OP_COMP_AND_SWAP, comp_flags, ah); break; case DAT_IB_FETCH_AND_ADD_OP: @@ -146,8 +148,23 @@ dapl_extensions(IN DAT_HANDLE dat_handle, status = dapli_post_ext(ep, dat_uint64a, 0, 0, segments, lmr_p, cookie, rmr_p, - OP_FETCH_AND_ADD, comp_flags ); + OP_FETCH_AND_ADD, comp_flags, ah); + break; + case DAT_IB_UD_SEND_OP: + dapl_dbg_log(DAPL_DBG_TYPE_RTN, + " UD post_send extension call\n"); + + ep = dat_handle; /* ep_handle */ + segments = va_arg( args, DAT_COUNT); /* segments */ + lmr_p = va_arg( args, DAT_LMR_TRIPLET*); + ah = va_arg( args, DAT_IB_ADDR_HANDLE*); + cookie = va_arg( args, DAT_DTO_COOKIE); + comp_flags = va_arg( args, DAT_COMPLETION_FLAGS); + + status = dapli_post_ext(ep, 0, 0, 0, segments, + lmr_p, cookie, NULL, + OP_SEND_UD, comp_flags, ah); break; default: @@ -169,7 +186,8 @@ dapli_post_ext( IN DAT_EP_HANDLE ep_handle, IN DAT_DTO_COOKIE user_cookie, IN const DAT_RMR_TRIPLET *remote_iov, IN int op_type, - IN DAT_COMPLETION_FLAGS flags ) + IN DAT_COMPLETION_FLAGS flags, + IN DAT_IB_ADDR_HANDLE *ah) { DAPL_EP *ep_ptr; ib_qp_handle_t qp_ptr; @@ -178,9 +196,9 @@ dapli_post_ext( IN DAT_EP_HANDLE ep_handle, dapl_dbg_log(DAPL_DBG_TYPE_API, " post_ext_op: ep %p cmp_val %d " - "swap_val %d cookie 0x%x, r_iov %p, flags 0x%x\n", + "swap_val %d cookie 0x%x, r_iov %p, flags 0x%x, ah %p\n", ep_handle, (unsigned)cmp_add, (unsigned)swap, - (unsigned)user_cookie.as_64, remote_iov, flags); + (unsigned)user_cookie.as_64, remote_iov, flags, ah); if (DAPL_BAD_HANDLE(ep_handle, DAPL_MAGIC_EP)) return(DAT_ERROR(DAT_INVALID_HANDLE, DAT_INVALID_HANDLE_EP)); @@ -217,7 +235,8 @@ dapli_post_ext( IN DAT_EP_HANDLE ep_handle, immed_data, /* immed data */ cmp_add, /* compare or add */ swap, /* swap */ - flags); + flags, + ah); if (dat_status != DAT_SUCCESS) { dapl_os_atomic_dec(&ep_ptr->req_count); @@ -288,6 +307,15 @@ dapls_cqe_to_event_extension(IN DAPL_EP *ep_ptr, ext_data->type = DAT_IB_RDMA_WRITE_IMMED_DATA; ext_data->val.immed.data = DAPL_GET_CQE_IMMED_DATA(cqe_ptr); break; + case OP_RECEIVE_MSG_IMM: + dapl_dbg_log (DAPL_DBG_TYPE_EVD, + " cqe_to_event_ext: OP_RECEIVE_MSG_IMMED\n"); + + /* immed recvd, type and inbound recv message transfer size */ + dto->transfered_length = DAPL_GET_CQE_BYTESNUM(cqe_ptr); + ext_data->type = DAT_IB_RECV_IMMED_DATA; + ext_data->val.immed.data = DAPL_GET_CQE_IMMED_DATA(cqe_ptr); + break; case OP_COMP_AND_SWAP: dapl_dbg_log (DAPL_DBG_TYPE_EVD, " cqe_to_event_ext: COMP_AND_SWAP_RESP\n"); @@ -304,6 +332,23 @@ dapls_cqe_to_event_extension(IN DAPL_EP *ep_ptr, ext_data->type = DAT_IB_FETCH_AND_ADD; dto->transfered_length = DAPL_GET_CQE_BYTESNUM(cqe_ptr); break; + case OP_SEND_UD: + dapl_dbg_log (DAPL_DBG_TYPE_EVD, + " cqe_to_event_ext: UD_SEND\n"); + + /* type and outbound send transfer size */ + ext_data->type = DAT_IB_UD_SEND; + dto->transfered_length = cookie->val.dto.size; + break; + case OP_RECV_UD: + dapl_dbg_log (DAPL_DBG_TYPE_EVD, + " cqe_to_event_ext: UD_RECV\n"); + + /* type and inbound recv message transfer size */ + ext_data->type = DAT_IB_UD_RECV; + dto->transfered_length = DAPL_GET_CQE_BYTESNUM(cqe_ptr); + break; + default: /* not extended operation */ ext_data->status = DAT_IB_OP_ERR; diff --git a/dapl/openib_scm/dapl_ib_qp.c b/dapl/openib_scm/dapl_ib_qp.c index 8577131..4fae307 100644 --- a/dapl/openib_scm/dapl_ib_qp.c +++ b/dapl/openib_scm/dapl_ib_qp.c @@ -114,6 +114,16 @@ dapls_ib_qp_alloc ( qp_create.cap.max_send_sge = attr->max_request_iov; qp_create.cap.max_inline_data = ia_ptr->hca_ptr->ib_trans.max_inline_send; qp_create.qp_type = IBV_QPT_RC; + +#ifdef DAT_EXTENSIONS + if (attr->service_type == DAT_IB_SERVICE_TYPE_UD) { + qp_create.qp_type = IBV_QPT_UD; + if (attr->max_message_size > + (128 << ia_ptr->hca_ptr->ib_trans.mtu)) { + return (DAT_INVALID_PARAMETER | DAT_INVALID_ARG6); + } + } +#endif qp_create.qp_context = (void*)ep_ptr; /* ibv assumes rcv_cq is never NULL, set to req_cq */ @@ -299,12 +309,13 @@ dapls_ib_reinit_ep ( DAT_RETURN dapls_modify_qp_state ( IN ib_qp_handle_t qp_handle, IN ib_qp_state_t qp_state, - IN ib_qp_cm_t *qp_cm ) + IN struct ib_cm_handle *cm_ptr ) { struct ibv_qp_attr qp_attr; enum ibv_qp_attr_mask mask = IBV_QP_STATE; DAPL_EP *ep_ptr = (DAPL_EP*)qp_handle->qp_context; DAPL_IA *ia_ptr = ep_ptr->header.owner_ia; + ib_qp_cm_t *qp_cm = &cm_ptr->dst; dapl_os_memzero((void*)&qp_attr, sizeof(qp_attr)); qp_attr.qp_state = qp_state; @@ -313,6 +324,12 @@ dapls_modify_qp_state ( IN ib_qp_handle_t qp_handle, /* additional attributes with RTR and RTS */ case IBV_QPS_RTR: { + dapl_dbg_log(DAPL_DBG_TYPE_EP, + " QPS_RTR: type %d qpn %x lid %x" + " port %x\n", + qp_handle->qp_type, + qp_cm->qpn, qp_cm->lid, qp_cm->port); + mask |= IBV_QP_AV | IBV_QP_PATH_MTU | IBV_QP_DEST_QPN | @@ -329,46 +346,59 @@ dapls_modify_qp_state ( IN ib_qp_handle_t qp_handle, ep_ptr->param.ep_attr.max_rdma_read_out; qp_attr.min_rnr_timer = ia_ptr->hca_ptr->ib_trans.rnr_timer; + + /* address handle */ qp_attr.ah_attr.dlid = qp_cm->lid; - /* global routing */ if (ia_ptr->hca_ptr->ib_trans.global) { qp_attr.ah_attr.is_global = 1; qp_attr.ah_attr.grh.dgid = qp_cm->gid; qp_attr.ah_attr.grh.hop_limit = - ia_ptr->hca_ptr->ib_trans.hop_limit; + ia_ptr->hca_ptr->ib_trans.hop_limit; qp_attr.ah_attr.grh.traffic_class = - ia_ptr->hca_ptr->ib_trans.tclass; + ia_ptr->hca_ptr->ib_trans.tclass; } qp_attr.ah_attr.sl = 0; qp_attr.ah_attr.src_path_bits = 0; - qp_attr.ah_attr.port_num = qp_cm->port; - - dapl_dbg_log (DAPL_DBG_TYPE_EP, - " modify_qp_rtr: qpn %x lid %x " - "port %x rd_atomic %d\n", - qp_cm->qpn, qp_cm->lid, qp_cm->port, - qp_attr.max_dest_rd_atomic ); - + qp_attr.ah_attr.port_num = ia_ptr->hca_ptr->port_num; +#ifdef DAT_EXTENSIONS + /* UD: create AH for remote side */ + if (qp_handle->qp_type == IBV_QPT_UD) { + ib_pd_handle_t pz; + pz = ((DAPL_PZ *) + ep_ptr->param.pz_handle)->pd_handle; + mask = IBV_QP_STATE; + cm_ptr->ah = ibv_create_ah(pz, + &qp_attr.ah_attr); + if (!cm_ptr->ah) + return(dapl_convert_errno(errno, + "ibv_ah")); + } +#endif break; } case IBV_QPS_RTS: { - mask |= IBV_QP_TIMEOUT | - IBV_QP_RETRY_CNT | - IBV_QP_RNR_RETRY | - IBV_QP_SQ_PSN | - IBV_QP_MAX_QP_RD_ATOMIC; - + mask |= IBV_QP_SQ_PSN; + if (qp_handle->qp_type == IBV_QPT_RC) { + mask |= IBV_QP_TIMEOUT | + IBV_QP_RETRY_CNT | + IBV_QP_RNR_RETRY | + IBV_QP_MAX_QP_RD_ATOMIC; + + qp_attr.timeout = + ia_ptr->hca_ptr->ib_trans.ack_timer; + qp_attr.retry_cnt = + ia_ptr->hca_ptr->ib_trans.ack_retry; + qp_attr.rnr_retry = + ia_ptr->hca_ptr->ib_trans.rnr_retry; + qp_attr.max_rd_atomic = + ep_ptr->param.ep_attr.max_rdma_read_out; + } qp_attr.qp_state = IBV_QPS_RTS; - qp_attr.timeout = ia_ptr->hca_ptr->ib_trans.ack_timer; - qp_attr.retry_cnt = ia_ptr->hca_ptr->ib_trans.ack_retry; - qp_attr.rnr_retry = ia_ptr->hca_ptr->ib_trans.rnr_retry; qp_attr.sq_psn = 1; - qp_attr.max_rd_atomic = - ep_ptr->param.ep_attr.max_rdma_read_out; dapl_dbg_log(DAPL_DBG_TYPE_EP, - " modify_qp_rts: psn %x rd_atomic %d ack %d " + " QPS_RTS: psn %x rd_atomic %d ack %d " " retry %d rnr_retry %d\n", qp_attr.sq_psn, qp_attr.max_rd_atomic, qp_attr.timeout, qp_attr.retry_cnt, @@ -377,23 +407,29 @@ dapls_modify_qp_state ( IN ib_qp_handle_t qp_handle, } case IBV_QPS_INIT: { - mask |= IBV_QP_PKEY_INDEX | - IBV_QP_PORT | - IBV_QP_ACCESS_FLAGS; - - qp_attr.pkey_index = 0; - qp_attr.port_num = ia_ptr->hca_ptr->port_num; - qp_attr.qp_access_flags = + mask |= IBV_QP_PKEY_INDEX | IBV_QP_PORT; + if (qp_handle->qp_type == IBV_QPT_RC) { + mask |= IBV_QP_ACCESS_FLAGS; + qp_attr.qp_access_flags = IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ | IBV_ACCESS_REMOTE_ATOMIC | IBV_ACCESS_MW_BIND; - + } +#ifdef DAT_EXTENSIONS + if (qp_handle->qp_type == IBV_QPT_UD) { + mask |= IBV_QP_QKEY; + qp_attr.qkey = SCM_UD_QKEY; + } +#endif + qp_attr.pkey_index = 0; + qp_attr.port_num = ia_ptr->hca_ptr->port_num; + dapl_dbg_log (DAPL_DBG_TYPE_EP, - " modify_qp_init: pi %x port %x acc %x\n", + " QPS_INIT: pi %x port %x acc %x qkey 0x%x\n", qp_attr.pkey_index, qp_attr.port_num, - qp_attr.qp_access_flags ); + qp_attr.qp_access_flags, qp_attr.qkey); break; } default: diff --git a/dapl/openib_scm/dapl_ib_util.c b/dapl/openib_scm/dapl_ib_util.c index 362710d..43f85ac 100644 --- a/dapl/openib_scm/dapl_ib_util.c +++ b/dapl/openib_scm/dapl_ib_util.c @@ -436,7 +436,7 @@ DAT_RETURN dapls_ib_query_hca ( ia_attr->max_lmr_virtual_address = dev_attr.max_mr_size; ia_attr->max_rmr_target_address = dev_attr.max_mr_size; ia_attr->max_pzs = dev_attr.max_pd; - ia_attr->max_mtu_size = port_attr.max_msg_sz; + ia_attr->max_message_size = port_attr.max_msg_sz; ia_attr->max_rdma_size = port_attr.max_msg_sz; ia_attr->max_iov_segments_per_rdma_read = dev_attr.max_sge; ia_attr->max_iov_segments_per_rdma_write = dev_attr.max_sge; @@ -463,14 +463,14 @@ DAT_RETURN dapls_ib_query_hca ( ia_attr->max_evds, ia_attr->max_evd_qlen ); dapl_dbg_log (DAPL_DBG_TYPE_UTIL, " query_hca: msg %llu rdma %llu iov %d lmr %d rmr %d ack_time %d\n", - ia_attr->max_mtu_size, ia_attr->max_rdma_size, + ia_attr->max_message_size, ia_attr->max_rdma_size, ia_attr->max_iov_segments_per_dto, ia_attr->max_lmrs, ia_attr->max_rmrs,hca_ptr->ib_trans.ack_timer ); } if (ep_attr != NULL) { (void) dapl_os_memzero(ep_attr, sizeof(*ep_attr)); - ep_attr->max_mtu_size = port_attr.max_msg_sz; + ep_attr->max_message_size = port_attr.max_msg_sz; ep_attr->max_rdma_size = port_attr.max_msg_sz; ep_attr->max_recv_dtos = dev_attr.max_qp_wr; ep_attr->max_request_dtos = dev_attr.max_qp_wr; @@ -479,8 +479,9 @@ DAT_RETURN dapls_ib_query_hca ( ep_attr->max_rdma_read_in = dev_attr.max_qp_rd_atom; ep_attr->max_rdma_read_out= dev_attr.max_qp_rd_atom; dapl_dbg_log (DAPL_DBG_TYPE_UTIL, - " query_hca: MAX msg %llu dto %d iov %d rdma i%d,o%d\n", - ep_attr->max_mtu_size, + " query_hca: MAX msg %llu mtu %d dto %d iov %d" + " rdma i%d,o%d\n", + ep_attr->max_message_size, ep_attr->max_recv_dtos, ep_attr->max_recv_iov, ep_attr->max_rdma_read_in, ep_attr->max_rdma_read_out); } @@ -574,6 +575,9 @@ DAT_NAMED_ATTR ib_attrs[] = { { DAT_IB_ATTR_IMMED_DATA, "TRUE" }, + { + DAT_IB_ATTR_UD, "TRUE" + }, #endif }; diff --git a/dapl/openib_scm/dapl_ib_util.h b/dapl/openib_scm/dapl_ib_util.h index 39eb245..bd3ea83 100644 --- a/dapl/openib_scm/dapl_ib_util.h +++ b/dapl/openib_scm/dapl_ib_util.h @@ -52,6 +52,10 @@ #include #include +#ifdef DAT_EXTENSIONS +#include +#endif + #ifndef __cplusplus #define false 0 #define true 1 @@ -72,7 +76,7 @@ typedef ib_hca_handle_t dapl_ibal_ca_t; /* CM mappings, user CM not complete use SOCKETS */ /* destination info to exchange, define wire protocol version */ -#define DSCM_VER 2 +#define DSCM_VER 3 typedef struct _ib_qp_cm { uint16_t ver; @@ -83,6 +87,7 @@ typedef struct _ib_qp_cm uint32_t p_size; DAT_SOCK_ADDR6 ia_address; union ibv_gid gid; + uint16_t qp_type; } ib_qp_cm_t; /* @@ -117,10 +122,11 @@ struct ib_cm_handle SCM_STATE state; int socket; struct dapl_hca *hca; - DAT_HANDLE sp; + struct dapl_sp *sp; struct dapl_ep *ep; ib_qp_cm_t dst; unsigned char p_data[256]; + struct ibv_ah *ah; }; typedef struct ib_cm_handle *dp_ib_cm_handle_t; @@ -180,6 +186,9 @@ typedef struct ibv_comp_channel *ib_wait_obj_handle_t; /* inline send rdma threshold */ #define INLINE_SEND_DEFAULT 128 +/* qkey for UD QP's */ +#define SCM_UD_QKEY 0x78654321 + /* RC timer - retry count defaults */ #define SCM_ACK_TIMER 15 /* 5 bits, 4.096us*2^ack_timer. 15 == 134ms */ #define SCM_ACK_RETRY 7 /* 3 bits, 7 * 134ms = 940ms */ @@ -207,8 +216,11 @@ typedef struct ibv_comp_channel *ib_wait_obj_handle_t; #define OP_COMP_AND_SWAP IBV_WR_ATOMIC_CMP_AND_SWP #define OP_FETCH_AND_ADD IBV_WR_ATOMIC_FETCH_AND_ADD #define OP_RECEIVE 7 /* internal op */ -#define OP_RECEIVE_IMM 8 /* internel op */ -#define OP_BIND_MW 9 /* internal op */ +#define OP_RECEIVE_IMM 8 /* rdma write with immed, internel op */ +#define OP_RECEIVE_MSG_IMM 9 /* recv msg with immed, internel op */ +#define OP_BIND_MW 10 /* internal op */ +#define OP_SEND_UD 11 /* internal op */ +#define OP_RECV_UD 12 /* internal op */ #define OP_INVALID 0xff /* Definitions to map QP state */ @@ -321,7 +333,7 @@ void dapli_cq_thread_destroy(struct dapl_hca *hca_ptr); DAT_RETURN dapls_modify_qp_state ( IN ib_qp_handle_t qp_handle, IN ib_qp_state_t qp_state, - IN ib_qp_cm_t *qp_cm ); + IN struct ib_cm_handle *cm_ptr ); /* inline functions */ STATIC _INLINE_ IB_HCA_NAME dapl_ib_convert_name (IN char *name) -- 1.5.2.5 From arlin.r.davis at intel.com Sun Jul 20 15:19:52 2008 From: arlin.r.davis at intel.com (Arlin Davis) Date: Sun, 20 Jul 2008 15:19:52 -0700 Subject: [ofa-general] [PATCH 4/4] uDAPL v2 - IB UD extension - dtestx: add -d option to test new IB UD extension Message-ID: <000301c8eab6$bccf0350$67bd020a@amr.corp.intel.com> modify dtestx to connect peer UD QP's and exchange/verify messages as an example for new extension. Also add -b buf_size, -h server host, and -P provider name options to match dtest semantics. Signed-off by: Arlin Davis ardavis at ichips.intel.com --- test/dtest/dtestx.c | 177 +++++++++++++++++++++++++++++++++++--------------- 1 files changed, 124 insertions(+), 53 deletions(-) diff --git a/test/dtest/dtestx.c b/test/dtest/dtestx.c index 1db60eb..fc3fc45 100755 --- a/test/dtest/dtestx.c +++ b/test/dtest/dtestx.c @@ -153,9 +153,22 @@ DAT_EVD_HANDLE con_evd = DAT_HANDLE_NULL; DAT_EVD_HANDLE dto_evd = DAT_HANDLE_NULL; DAT_PSP_HANDLE psp = DAT_HANDLE_NULL; DAT_CR_HANDLE cr = DAT_HANDLE_NULL; -int server; - -char *usage = "-s | hostname (default == -s)\n"; +int server = 1; +int ud_test = 0; +int buf_size = BUF_SIZE; +int msg_size = sizeof(DAT_RMR_TRIPLET); +char provider[64] = DAPL_PROVIDER; +DAT_IB_ADDR_HANDLE remote_ah; + +void print_usage(void) +{ + printf("\n dtestx usage \n\n"); + printf("u unreliable datagram test\n"); + printf("b: buf length to allocate\n"); + printf("h: hostname/address of server, specified on client\n"); + printf("P: provider name (default = ofa-v2-ib0)\n"); + printf("\n"); +} #if defined(_WIN32) || defined(_WIN64) static void sleep(int secs) @@ -186,24 +199,25 @@ send_msg( iov.virtual_address = (DAT_VADDR)data; iov.segment_length = (DAT_VLEN)size; - status = dat_ep_post_send(ep, - 1, - &iov, - cookie, - flags); + if (ud_test) + status = dat_ib_post_send_ud(ep, 1, &iov, + &remote_ah, cookie, flags); + else + status = dat_ep_post_send(ep, 1, &iov, cookie, flags); + _OK(status, "dat_ep_post_send"); - if (! (flags & DAT_COMPLETION_SUPPRESS_FLAG)) { + if (!(flags & DAT_COMPLETION_SUPPRESS_FLAG)) { status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore); _OK(status, "dat_evd_wait after dat_ep_post_send"); - if (event.event_number != DAT_DTO_COMPLETION_EVENT) { + if ((event.event_number != DAT_DTO_COMPLETION_EVENT) && + (ud_test && event.event_number != DAT_IB_DTO_EVENT)) { printf("unexpected event waiting for post_send " "completion - 0x%x\n", event.event_number); exit(1); } - _OK(dto_event->status, "event status for post_send"); } } @@ -225,8 +239,11 @@ connect_ep(char *hostname) &event.event_data.cr_arrival_event_data; DAT_DTO_COMPLETION_EVENT_DATA *dto_event = &event.event_data.dto_completion_event_data; + DAT_IB_EXTENSION_EVENT_DATA *ext_event = + (DAT_IB_EXTENSION_EVENT_DATA *) + &event.event_extension_data[0]; - status = dat_ia_open(DAPL_PROVIDER, 8, &async_evd, &ia); + status = dat_ia_open(provider, 8, &async_evd, &ia); _OK(status, "dat_ia_open"); status = dat_pz_create(ia, &pz); @@ -243,16 +260,24 @@ connect_ep(char *hostname) _OK(status, "dat_evd_create DTO"); memset(&ep_attr, 0, sizeof(ep_attr)); - ep_attr.service_type = DAT_SERVICE_TYPE_RC; - ep_attr.max_rdma_size = 0x10000; + if (ud_test) { + msg_size += 40; + ep_attr.service_type = DAT_IB_SERVICE_TYPE_UD; + ep_attr.max_message_size = buf_size; + ep_attr.max_rdma_read_in = 0; + ep_attr.max_rdma_read_out = 0; + } else { + ep_attr.service_type = DAT_SERVICE_TYPE_RC; + ep_attr.max_rdma_size = 0x10000; + ep_attr.max_rdma_read_in = 4; + ep_attr.max_rdma_read_out = 4; + } ep_attr.qos = 0; ep_attr.recv_completion_flags = 0; ep_attr.max_recv_dtos = 10; ep_attr.max_request_dtos = 10; ep_attr.max_recv_iov = 1; ep_attr.max_request_iov = 1; - ep_attr.max_rdma_read_in = 4; - ep_attr.max_rdma_read_out = 4; ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG; ep_attr.ep_transport_specific_count = 0; ep_attr.ep_transport_specific = NULL; @@ -264,12 +289,12 @@ connect_ep(char *hostname) _OK(status, "dat_ep_create"); for (i = 0; i < REG_MEM_COUNT; i++) { - buf[ i ] = (DAT_RMR_TRIPLET*)malloc(BUF_SIZE); + buf[ i ] = (DAT_RMR_TRIPLET*)malloc(buf_size); region.for_va = buf[ i ]; status = dat_lmr_create(ia, DAT_MEM_TYPE_VIRTUAL, region, - BUF_SIZE, + buf_size, pz, DAT_MEM_PRIV_ALL_FLAG| DAT_IB_MEM_PRIV_REMOTE_ATOMIC, @@ -283,7 +308,7 @@ connect_ep(char *hostname) } /* register atomic return buffer for original data */ - atomic_buf = (DAT_UINT64*)malloc(BUF_SIZE); + atomic_buf = (DAT_UINT64*)malloc(BUF_SIZE_ATOMIC); region.for_va = atomic_buf; status = dat_lmr_create(ia, DAT_MEM_TYPE_VIRTUAL, @@ -304,7 +329,7 @@ connect_ep(char *hostname) cookie.as_64 = i; iov.lmr_context = lmr_context[ i ]; iov.virtual_address = (DAT_VADDR) buf[ i ]; - iov.segment_length = BUF_SIZE; + iov.segment_length = buf_size; status = dat_ep_post_recv(ep, 1, @@ -330,9 +355,11 @@ connect_ep(char *hostname) status = dat_evd_wait(cr_evd, SERVER_TIMEOUT, 1, &event, &nmore); _OK(status, "listen dat_evd_wait"); - - if (event.event_number != DAT_CONNECTION_REQUEST_EVENT) { - printf("unexpected event after dat_psp_create: 0x%x\n", + + if (event.event_number != DAT_CONNECTION_REQUEST_EVENT && + (ud_test && event.event_number != + DAT_IB_UD_CONNECTION_REQUEST_EVENT)) { + printf("unexpected event, !conn request: 0x%x\n", event.event_number); exit(1); } @@ -377,22 +404,39 @@ connect_ep(char *hostname) status = dat_evd_wait(con_evd, CONN_TIMEOUT, 1, &event, &nmore); _OK(status, "connect dat_evd_wait"); - if (event.event_number != DAT_CONNECTION_EVENT_ESTABLISHED) { - printf("unexpected event after dat_ep_connect: 0x%x\n", + if (event.event_number != DAT_CONNECTION_EVENT_ESTABLISHED && + (ud_test && event.event_number != + DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) { + printf("unexpected event, !conn established: 0x%x\n", event.event_number); exit(1); } + /* if UD get the AH for the sends */ + if (ud_test) { + if (ext_event->type == DAT_IB_UD_REMOTE_AH) { + remote_ah = ext_event->remote_ah; + printf(" remote_ah: ah=%p, qpn=0x%x addr=%s\n", + remote_ah.ah, remote_ah.qpn, + inet_ntoa(((struct sockaddr_in *) + &remote_ah.ia_addr)->sin_addr)); + } else { + printf("unexpected UD ext_event type: 0x%x\n", + ext_event->type); + exit(1); + } + } + printf("Connected!\n"); /* * Setup our remote memory and tell the other side about it * Swap to network order. */ - r_iov = (DAT_RMR_TRIPLET*)buf[SEND_BUF_INDEX]; - r_iov->rmr_context = hton32(rmr_context[RCV_RDMA_BUF_INDEX]); + r_iov = (DAT_RMR_TRIPLET*)buf[SEND_BUF_INDEX]; + r_iov->rmr_context = hton32(rmr_context[RCV_RDMA_BUF_INDEX]); r_iov->virtual_address = hton64((DAT_VADDR)buf[RCV_RDMA_BUF_INDEX]); - r_iov->segment_length = hton32(BUF_SIZE); + r_iov->segment_length = hton32(buf_size); printf("%d Send RMR msg to remote: r_key_ctx=0x%x,va=%p,len=0x%x\n", getpid(), hton32(r_iov->rmr_context), @@ -411,25 +455,30 @@ connect_ep(char *hostname) status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore); _OK(status, "dat_evd_wait after dat_ep_post_send"); - if (event.event_number != DAT_DTO_COMPLETION_EVENT) { - printf("unexpected event waiting for RMR context - 0x%x\n", - event.event_number); + if ((event.event_number != DAT_DTO_COMPLETION_EVENT) && + (ud_test && event.event_number != DAT_IB_DTO_EVENT)) { + printf("unexpected event waiting for RMR context " + "- 0x%x\n", event.event_number); exit(1); } + _OK(dto_event->status, "event status for post_recv"); - _OK(dto_event->status, "event status for post_send"); - if ((dto_event->transfered_length != sizeof(DAT_RMR_TRIPLET)) || - (dto_event->user_cookie.as_64 != RECV_BUF_INDEX)) { + if (dto_event->transfered_length != msg_size || + dto_event->user_cookie.as_64 != RECV_BUF_INDEX) { printf("unexpected event data for receive: len=%d cookie=%d " "expected %d/%d\n", (int)dto_event->transfered_length, (int)dto_event->user_cookie.as_64, - sizeof(DAT_RMR_TRIPLET), RECV_BUF_INDEX); + msg_size, RECV_BUF_INDEX); exit(1); } /* swap RMR,address info to host order */ - r_iov = (DAT_RMR_TRIPLET*)buf[RECV_BUF_INDEX]; + if (ud_test) + r_iov = (DAT_RMR_TRIPLET*)((char*)buf[RECV_BUF_INDEX]+40); + else + r_iov = (DAT_RMR_TRIPLET*)buf[RECV_BUF_INDEX]; + r_iov->rmr_context = ntoh32(r_iov->rmr_context); r_iov->virtual_address = ntoh64(r_iov->virtual_address); r_iov->segment_length = ntoh32(r_iov->segment_length); @@ -525,7 +574,7 @@ do_immediate() iov.lmr_context = lmr_context[SND_RDMA_BUF_INDEX]; iov.virtual_address = (DAT_VADDR) buf[SND_RDMA_BUF_INDEX]; - iov.segment_length = BUF_SIZE; + iov.segment_length = buf_size; cookie.as_64 = 0x9999; @@ -554,7 +603,7 @@ do_immediate() _OK(dto_event->status, "DTO event status"); if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED) { - if ((dto_event->transfered_length != BUF_SIZE) || + if ((dto_event->transfered_length != buf_size) || (dto_event->user_cookie.as_64 != 0x9999)) { printf("unexpected event data for rdma_write_immed: len=%d " @@ -566,7 +615,7 @@ do_immediate() } else if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED_DATA) { - if ((dto_event->transfered_length != BUF_SIZE) || + if ((dto_event->transfered_length != buf_size) || (dto_event->user_cookie.as_64 != RECV_BUF_INDEX+1)) { printf("unexpected event data of immediate write: len=%d " @@ -603,7 +652,7 @@ do_immediate() _OK(dto_event->status, "event status"); if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED) { - if ((dto_event->transfered_length != BUF_SIZE) || + if ((dto_event->transfered_length != buf_size) || (dto_event->user_cookie.as_64 != 0x9999)) { printf("unexpected event data for rdma_write_immed: len=%d " @@ -615,7 +664,7 @@ do_immediate() } else if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED_DATA) { - if ((dto_event->transfered_length != BUF_SIZE) || + if ((dto_event->transfered_length != buf_size) || (dto_event->user_cookie.as_64 != RECV_BUF_INDEX+1)) { printf("unexpected event data of immediate write: len=%d " @@ -904,25 +953,39 @@ do_fetch_add() return(0); } + int main(int argc, char **argv) { char *hostname; - int rc=0; - - if (argc > 2) { - printf(usage); - exit(1); - } - - if ((argc == 1) || strcmp(argv[ 1 ], "-s") == 0) + int rc; + + /* parse arguments */ + while ((rc = getopt(argc, argv, "uh:b:P:")) != -1) { - server = 1; - } else { - server = 0; - hostname = argv[ 1 ]; + switch(rc) + { + case 'u': + ud_test = 1; + fflush(stdout); + break; + case 'h': + server = 0; + strcpy (hostname, optarg); + break; + case 'b': + buf_size = atoi(optarg); + break; + case 'P': + strcpy (provider, optarg); + break; + default: + print_usage(); + exit(-12); + } } + #if defined(_WIN32) || defined(_WIN64) { WSADATA wsaData; @@ -944,6 +1007,9 @@ main(int argc, char **argv) _WSACleanup(); exit(1); } + if (ud_test) + goto bail; + if (do_immediate()) { _WSACleanup(); exit(1); @@ -956,7 +1022,12 @@ main(int argc, char **argv) _WSACleanup(); exit(1); } +bail: rc = disconnect_ep(); _WSACleanup(); + + if (!rc) + printf("\n IB extension test - %s test PASSED\n\n", + ud_test?"UD":"immed/atomic"); return rc; } -- 1.5.2.5 From ogerlitz at voltaire.com Sun Jul 20 22:04:32 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 21 Jul 2008 08:04:32 +0300 Subject: [ofa-general] InfiniBand/RDMA merge plans for 2.6.27 In-Reply-To: References: Message-ID: <488418E0.1080100@voltaire.com> Roland Dreier wrote: > - I'm waiting to merge the RDMA_CM_EVENT_ADDR_CHANGE changes that > depend on core networking changes until such changes are upstream. > Or, please remind me when that happens. the net changes are now merged into Linus tree Or. From 7bxnvlb at daufuskieresort.com Sun Jul 20 20:40:10 2008 From: 7bxnvlb at daufuskieresort.com (clemmy ikonas) Date: Mon, 21 Jul 2008 03:40:10 +0000 Subject: [ofa-general] to general Message-ID: <000801c8eaf2$067f238b$27ada6ba@ypciyp> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.sawwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From jackm at dev.mellanox.co.il Mon Jul 21 00:11:03 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 21 Jul 2008 10:11:03 +0300 Subject: [ofa-general] [PATCH 1/3] libibverbs : add new verbs: create_qp_expanded In-Reply-To: References: Message-ID: <200807211011.04098.jackm@dev.mellanox.co.il> On Sunday 20 July 2008 22:14, Ron Livne wrote: > index f8138ef..47ec759 100644 > --- a/include/infiniband/driver.h > +++ b/include/infiniband/driver.h > @@ -155,6 +155,11 @@ int ibv_cmd_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, > uint32_t xrc_qp_num); > int ibv_cmd_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, > uint32_t xrc_qp_num); > +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, > + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, > + uint32_t create_flags, > + struct ibv_create_qp *cmd, size_t cmd_size, > + struct ibv_create_qp_resp *resp, size_t resp_size); > You should be defining new structures: struct ibv_create_qp_expanded and struct ibv_create_qp_expanded_resp (Possibly only the first is needed). > > /* > @@ -455,7 +456,7 @@ struct ibv_create_qp { > __u8 sq_sig_all; > __u8 qp_type; > __u8 is_srq; > - __u8 reserved; > + __u8 create_flags; > __u64 driver_data[0]; > }; You should not need to touch struct ibv_create_qp You should define struct ibv_create_qp_expanded { ... __u8 sq_sig_all; __u8 qp_type; __u8 is_srq; __u8 reserved; __u32 create_flags; __u32 reserved2; __u64 driver_data[0]; }; and use that all along the way. > diff --git a/src/cmd.c b/src/cmd.c > index cedf55e..802f3ef 100644 > --- a/src/cmd.c > +++ b/src/cmd.c > @@ -642,7 +642,68 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, > cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? > (attr->xrc_domain ? attr->xrc_domain->handle : 0) : > (attr->srq ? attr->srq->handle : 0); > - cmd->reserved = 0; > + cmd->create_flags = 0; > + > + if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) > + return errno; > + > + VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); > + > + qp->handle = resp->qp_handle; > + qp->qp_num = resp->qpn; > + qp->context = pd->context; > + > + if (abi_ver > 3) { > + attr->cap.max_recv_sge = resp->max_recv_sge; > + attr->cap.max_send_sge = resp->max_send_sge; > + attr->cap.max_recv_wr = resp->max_recv_wr; > + attr->cap.max_send_wr = resp->max_send_wr; > + attr->cap.max_inline_data = resp->max_inline_data; > + } > + > + if (abi_ver == 4) { > + struct ibv_create_qp_resp_v4 *resp_v4 = > + (struct ibv_create_qp_resp_v4 *) resp; > + > + memmove((void *) resp + sizeof *resp, > + (void *) resp_v4 + sizeof *resp_v4, > + resp_size - sizeof *resp); > + } else if (abi_ver <= 3) { > + struct ibv_create_qp_resp_v3 *resp_v3 = > + (struct ibv_create_qp_resp_v3 *) resp; > + > + memmove((void *) resp + sizeof *resp, > + (void *) resp_v3 + sizeof *resp_v3, > + resp_size - sizeof *resp); > + } > + > + return 0; > +} > + > +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, > + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, > + uint32_t create_flags, > + struct ibv_create_qp *cmd, size_t cmd_size, > + struct ibv_create_qp_resp *resp, size_t resp_size) > +{ > + IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP, resp, resp_size); > + > + cmd->create_flags = create_flags; > + cmd->user_handle = (uintptr_t) qp; > + cmd->pd_handle = pd->handle; > + cmd->send_cq_handle = attr->send_cq->handle; > + cmd->recv_cq_handle = attr->recv_cq->handle; > + cmd->max_send_wr = attr->cap.max_send_wr; > + cmd->max_recv_wr = attr->cap.max_recv_wr; > + cmd->max_send_sge = attr->cap.max_send_sge; > + cmd->max_recv_sge = attr->cap.max_recv_sge; > + cmd->max_inline_data = attr->cap.max_inline_data; > + cmd->sq_sig_all = attr->sq_sig_all; > + cmd->qp_type = attr->qp_type; > + cmd->is_srq = !!attr->srq; > + cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? > + (attr->xrc_domain ? attr->xrc_domain->handle : 0) : > + (attr->srq ? attr->srq->handle : 0); > > if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) > return errno; You should be modifying ibv_cmd_create_qp so that it calls ibv_cmd_create_qp_expanded with the "create_flags" parameter = 0; This will allow you to shrink the code. > @@ -449,6 +449,32 @@ struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, > } > default_symver(__ibv_create_qp, ibv_create_qp); > > +struct ibv_qp *__ibv_create_qp_expanded(struct ibv_pd *pd, > + struct ibv_qp_init_attr *qp_init_attr, > + uint32_t create_flags) > +{ > + struct ibv_qp *qp = pd->context->more_ops->create_qp_expanded(pd, qp_init_attr, create_flags); > + > + if (qp) { > + qp->context = pd->context; > + qp->qp_context = qp_init_attr->qp_context; > + qp->pd = pd; > + qp->send_cq = qp_init_attr->send_cq; > + qp->recv_cq = qp_init_attr->recv_cq; > + qp->srq = qp_init_attr->srq; > + qp->qp_type = qp_init_attr->qp_type; > + qp->state = IBV_QPS_RESET; > + qp->events_completed = 0; > + qp->xrc_domain = qp_init_attr->qp_type == IBV_QPT_XRC ? > + qp_init_attr->xrc_domain : NULL; > + pthread_mutex_init(&qp->mutex, NULL); > + pthread_cond_init(&qp->cond, NULL); > + } > + > + return qp; > +} > +default_symver(__ibv_create_qp_expanded, ibv_create_qp_expanded); You don't need default_symver here, because this is a completely new verb. just use struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr, uint32_t create_flags) From jackm at dev.mellanox.co.il Mon Jul 21 00:13:44 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 21 Jul 2008 10:13:44 +0300 Subject: [ofa-general] [PATCH 2/3] libmlx4: add support for create_qp_expanded In-Reply-To: References: Message-ID: <200807211013.44251.jackm@dev.mellanox.co.il> You need to modify mlx4_create_qp() as well, so that it calls mlx4_create_qp_expanded with create_flags = 0 (and get rid of all the excess code). On Sunday 20 July 2008 22:15, Ron Livne wrote: > Adds support for the new verb: create_qp_expanded > in libmlx4. > > Signed-off-by: Ron Livne > > diff --git a/src/mlx4.c b/src/mlx4.c > index 1f23144..bfcb7ec 100644 > --- a/src/mlx4.c > +++ b/src/mlx4.c > @@ -79,6 +79,7 @@ static struct ibv_more_ops mlx4_more_ops = { > .query_xrc_rcv_qp = mlx4_query_xrc_rcv_qp, > .reg_xrc_rcv_qp = mlx4_reg_xrc_rcv_qp, > .unreg_xrc_rcv_qp = mlx4_unreg_xrc_rcv_qp, > + .create_qp_expanded = mlx4_create_qp_expanded, > #endif > }; > #endif > diff --git a/src/mlx4.h b/src/mlx4.h > index a2e3aec..7ed3ac8 100644 > --- a/src/mlx4.h > +++ b/src/mlx4.h > @@ -427,6 +427,9 @@ int mlx4_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, > uint32_t xrc_qp_num); > int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, > uint32_t xrc_qp_num); > +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, > + struct ibv_qp_init_attr *attr, > + uint32_t create_flags); > #endif > > > diff --git a/src/verbs.c b/src/verbs.c > index b1bad3c..9a99658 100644 > --- a/src/verbs.c > +++ b/src/verbs.c > @@ -825,4 +825,115 @@ int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, > return ibv_cmd_unreg_xrc_rcv_qp(xrc_domain, xrc_qp_num); > } > > +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, > + struct ibv_qp_init_attr *attr, > + uint32_t create_flags) > +{ > + struct mlx4_create_qp cmd; > + struct ibv_create_qp_resp resp; > + struct mlx4_qp *qp; > + int ret; > + > + /* Sanity check QP size before proceeding */ > + if (attr->cap.max_send_wr > 65536 || > + attr->cap.max_recv_wr > 65536 || > + attr->cap.max_send_sge > 64 || > + attr->cap.max_recv_sge > 64 || > + attr->cap.max_inline_data > 1024) > + return NULL; > + > + qp = malloc(sizeof *qp); > + if (!qp) > + return NULL; > + > + mlx4_calc_sq_wqe_size(&attr->cap, attr->qp_type, qp); > + > + /* > + * We need to leave 2 KB + 1 WQE of headroom in the SQ to > + * allow HW to prefetch. > + */ > + qp->sq_spare_wqes = (2048 >> qp->sq.wqe_shift) + 1; > + qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); > + qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); > + > + if (attr->srq || attr->qp_type == IBV_QPT_XRC) > + attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; > + else { > + if (attr->cap.max_recv_sge < 1) > + attr->cap.max_recv_sge = 1; > + if (attr->cap.max_recv_wr < 1) > + attr->cap.max_recv_wr = 1; > + } > + > + if (mlx4_alloc_qp_buf(pd, &attr->cap, attr->qp_type, qp)) > + goto err; > + > + mlx4_init_qp_indices(qp); > + > + if (pthread_spin_init(&qp->sq.lock, PTHREAD_PROCESS_PRIVATE) || > + pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) > + goto err_free; > + > + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { > + qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); > + if (!qp->db) > + goto err_free; > + > + *qp->db = 0; > + } > + > + cmd.buf_addr = (uintptr_t) qp->buf.buf; > + if (attr->srq || attr->qp_type == IBV_QPT_XRC) > + cmd.db_addr = 0; > + else > + cmd.db_addr = (uintptr_t) qp->db; > + cmd.log_sq_stride = qp->sq.wqe_shift; > + for (cmd.log_sq_bb_count = 0; > + qp->sq.wqe_cnt > 1 << cmd.log_sq_bb_count; > + ++cmd.log_sq_bb_count) > + ; /* nothing */ > + cmd.sq_no_prefetch = 0; > + > + ret = ibv_cmd_create_qp_expanded(pd, &qp->ibv_qp, attr, create_flags, > + &cmd.ibv_cmd, sizeof cmd, > + &resp, sizeof resp); > + if (ret) > + goto err_rq_db; > + > + ret = mlx4_store_qp(to_mctx(pd->context), qp->ibv_qp.qp_num, qp); > + if (ret) > + goto err_destroy; > + > + qp->rq.wqe_cnt = qp->rq.max_post = attr->cap.max_recv_wr; > + qp->rq.max_gs = attr->cap.max_recv_sge; > + mlx4_set_sq_sizes(qp, &attr->cap, attr->qp_type); > + > + qp->doorbell_qpn = htonl(qp->ibv_qp.qp_num << 8); > + if (attr->sq_sig_all) > + qp->sq_signal_bits = htonl(MLX4_WQE_CTRL_CQ_UPDATE); > + else > + qp->sq_signal_bits = 0; > + > + return &qp->ibv_qp; > + > +err_destroy: > + ibv_cmd_destroy_qp(&qp->ibv_qp); > + > +err_rq_db: > + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) > + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); > + > +err_free: > + free(qp->sq.wrid); > + if (qp->rq.wqe_cnt) > + free(qp->rq.wrid); > + mlx4_free_buf(&qp->buf); > + > +err: > + free(qp); > + > + return NULL; > +} > + > + > #endif > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From jackm at dev.mellanox.co.il Mon Jul 21 00:34:07 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 21 Jul 2008 10:34:07 +0300 Subject: [ofa-general] [PATCH 3/3] ib/uverbs: add support for create_qp_expanded in uverbs In-Reply-To: References: Message-ID: <200807211034.07393.jackm@dev.mellanox.co.il> In general, this should be written so that ib_uverbs_create_qp invokes a helper function which does work both for ib_uverbs_create_qp and ib_uverbs_create_qp_expanded. (ib_uverbs_create_qp() is only needed to support older libibverbs/libmlx4 libraries. The new library will do create-qp via create-qp-expanded, with create-flags = 0). - Jack On Sunday 20 July 2008 22:16, Ron Livne wrote: > @@ -705,8 +699,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, > IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)) > return ERR_PTR(-EINVAL); > > - if (init_attr->create_flags && > - (pd->uobject || init_attr->qp_type != IB_QPT_UD)) > + if (init_attr->create_flags && init_attr->qp_type != IB_QPT_UD) > return ERR_PTR(-EINVAL); You have opened the door here for userspace to set IB_QP_CREATE_IPOIB_UD_LSO This is incorrect. Please fix. > From julia at diku.dk Mon Jul 21 00:57:04 2008 From: julia at diku.dk (Julia Lawall) Date: Mon, 21 Jul 2008 09:57:04 +0200 (CEST) Subject: [ofa-general] [PATCH 1/7] drivers/infiniband: Release mutex in error handling code Message-ID: From: Julia Lawall The mutex is released on a successful return, so it would seem that it should be released on an error return as well. The semantic patch that makes this change is as follows: (http://www.emn.fr/x-info/coccinelle/) // @@ expression l; @@ mutex_lock(l); ... when != mutex_unlock(l) when any when strict ( if (...) { ... when != mutex_unlock(l) + mutex_unlock(l); return ...; } | mutex_unlock(l); ) // Signed-off-by: Julia Lawall --- diff -u -p a/drivers/infiniband/hw/ehca/ipz_pt_fn.c b/drivers/infiniband/hw/ehca/ipz_pt_fn.c --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c @@ -163,6 +163,7 @@ static int alloc_small_queue_page(struct out: ehca_err(pd->ib_pd.device, "failed to allocate small queue page"); + mutex_unlock(&pd->lock); return 0; } From vlad at lists.openfabrics.org Mon Jul 21 02:52:23 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Mon, 21 Jul 2008 02:52:23 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080721-0200 daily build status Message-ID: <20080721095223.180D8E6089A@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26-rc6 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Failed: Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26-rc6 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.26-rc6_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080721-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26-rc6 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26-rc6' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From fenkes at de.ibm.com Mon Jul 21 04:13:23 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 21 Jul 2008 13:13:23 +0200 Subject: [ofa-general] [PATCH 0/2] IB/ehca: Two minor circumventions Message-ID: <200807211313.23648.fenkes@de.ibm.com> [1/2] fixes spurious PATH_MIG events with certain FW versions [2/2] inserts a default value for Local CA ACK Delay Please review these patches and queue them for inclusion into the kernel if you think they're okay. Thanks! Joachim From fenkes at de.ibm.com Mon Jul 21 04:15:31 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 21 Jul 2008 13:15:31 +0200 Subject: [ofa-general] [PATCH 1/2] IB/ehca: Filter PATH_MIG events if QP was never armed In-Reply-To: <200807211313.23648.fenkes@de.ibm.com> References: <200807211313.23648.fenkes@de.ibm.com> Message-ID: <200807211315.31761.fenkes@de.ibm.com> Certain firmware versions sometimes cause spurious PATH_MIG events to occur during QP creation. Filter these events by making sure PATH_MIG events are only handed down when they actually make sense (i.e. when the QP has been armed at least once). Signed-off-by: Joachim Fenkes --- drivers/infiniband/hw/ehca/ehca_classes.h | 1 + drivers/infiniband/hw/ehca/ehca_irq.c | 4 ++++ drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++ 3 files changed, 7 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index 1e9e99a..0b0618e 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -194,6 +194,7 @@ struct ehca_qp { u32 packet_count; atomic_t nr_events; /* events seen */ wait_queue_head_t wait_completion; + int mig_armed; }; #define IS_SRQ(qp) (qp->ext_type == EQPT_SRQ) diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c index 0792d93..99642a6 100644 --- a/drivers/infiniband/hw/ehca/ehca_irq.c +++ b/drivers/infiniband/hw/ehca/ehca_irq.c @@ -178,6 +178,10 @@ static void dispatch_qp_event(struct ehca_shca *shca, struct ehca_qp *qp, { struct ib_event event; + /* PATH_MIG without the QP ever having been armed is false alarm */ + if (event_type == IB_EVENT_PATH_MIG && !qp->mig_armed) + return; + event.device = &shca->ib_device; event.event = event_type; diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index 3f59587..ea13efd 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -1460,6 +1460,8 @@ static int internal_modify_qp(struct ib_qp *ibqp, goto modify_qp_exit2; } mqpcb->path_migration_state = attr->path_mig_state + 1; + if (attr->path_mig_state == IB_MIG_REARM) + my_qp->mig_armed = 1; update_mask |= EHCA_BMASK_SET(MQPCB_MASK_PATH_MIGRATION_STATE, 1); } -- 1.5.5 From fenkes at de.ibm.com Mon Jul 21 04:16:36 2008 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Mon, 21 Jul 2008 13:16:36 +0200 Subject: [ofa-general] [PATCH 2/2] IB/ehca: Default value for Local CA ACK Delay In-Reply-To: <200807211313.23648.fenkes@de.ibm.com> References: <200807211313.23648.fenkes@de.ibm.com> Message-ID: <200807211316.37224.fenkes@de.ibm.com> Some firmware versions report a Local CA ACK Delay of 0. In that case, return a more sensible default value of 12 (-> 16 msec) instead. Signed-off-by: Joachim Fenkes --- drivers/infiniband/hw/ehca/ehca_hca.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c index bc3b37d..4628822 100644 --- a/drivers/infiniband/hw/ehca/ehca_hca.c +++ b/drivers/infiniband/hw/ehca/ehca_hca.c @@ -114,7 +114,9 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props) } props->max_pkeys = 16; - props->local_ca_ack_delay = min_t(u8, rblock->local_ca_ack_delay, 255); + /* Some FW versions say 0 here; insert sensible value in that case */ + props->local_ca_ack_delay = rblock->local_ca_ack_delay ? + min_t(u8, rblock->local_ca_ack_delay, 255) : 12; props->max_raw_ipv6_qp = limit_uint(rblock->max_raw_ipv6_qp); props->max_raw_ethy_qp = limit_uint(rblock->max_raw_ethy_qp); props->max_mcast_grp = limit_uint(rblock->max_mcast_grp); -- 1.5.5 From orenk at dev.mellanox.co.il Mon Jul 21 05:52:18 2008 From: orenk at dev.mellanox.co.il (Oren Kladnitsky) Date: Mon, 21 Jul 2008 15:52:18 +0300 Subject: [ofa-general] Re: "ibdiagnet -r" and zero systemguids In-Reply-To: <487E9528.4000202@hpc.ufl.edu> References: <487E9393.1080202@hpc.ufl.edu> <487E9528.4000202@hpc.ufl.edu> Message-ID: <48848682.40008@dev.mellanox.co.il> Craig Prescott wrote: > > I forgot to add that other than this > SystemGUID=0x0000000000000000 issue, the HCA appears > to work perfectly. > > Thanks, > Craig > > Craig Prescott wrote: >> >> Hi; >> >> When we run 'ibdiagnet -r' on our OFED 1.2 cluster, >> it bombs with a complaint about a system guid that is >> zero on our only PCI-X HCA in the fabric (see appended). >> ibdiagnet seems to be trying to saw off the leading zeroes >> from the system guid, and to have nothing left afterwards >> seems unexpected. >> >> Running 'ibdiagnet -r' from an OFED 1.3.1 machine does >> not bomb, but I am still concerned/unclear. >> >> My questions are: is it ok to have an HCA running >> around on your fabric with a system guid of zero? >> What if there was more than one? Is there any way to >> assign this HCA a sensible system guid, and would it >> be useful? >> >> The HCA in question is a Cougar cub running the 3.5.0 >> firmware from Mellanox. FWIW, the node and port guids >> for this HCA look sensible: >> >> [root at submit ~]# tvflash -g >> HCA #0 >> Node GUID = 0005ad0000050948 >> Port1 GUID = 0005ad0000050949 >> Port2 GUID = 0005ad000005094a >> >> If it isn't obvious already, I confess I'm not clear >> about how system guids are used. From what I can gather >> from google-ing around, a system guid of zero for an HCA >> means that the HCA vendor simply did not assign one. I >> am under the impression that this is uncommon, but not >> unheard of. Is that correct? >> >> I did some searches through both volumes of the 1.2.1 IB >> spec and came up empty, but I could have easily missed any >> substantial discussion about system guids. Any pointers or >> enlightenment in this area would be appreciated. >> >> Thanks, >> Craig Prescott >> UF HPC Center >> >> [root at submit ~]# ibdiagnet -r >> Loading IBDIAGNET from: /usr/lib64/ibdiagnet1.2 >> Loading IBDM from: /usr/lib64/ibdm1.2 >> -W- Topology file is not specified. >> Reports regarding cluster links will use direct routes. >> -I- Using port 1 as the local port. >> -I- Discovering the subnet ... 394 nodes (46 Switches & 348 CA-s) >> discovered. >> >> -I- Parsing Subnet file:/tmp/ibdiagnet.lst >> -I- Defined 382/394 systems/nodes >> >> -I--------------------------------------------------- >> -I- Bad Guids Info >> -I--------------------------------------------------- >> -W- Found Device with SystemGUID=0x0000000000000000: >> a HCA The Local Device "submit.ufhpc/P1" >> PortGUID=0x0005ad0000050949 at direct path="" >> ... >> -I--------------------------------------------------- >> -I- mgid-mlid-HCAs matching table >> -I--------------------------------------------------- >> mgid | mlid | HCAs >> -------------------------------------------------------------------------------- >> >> >> >> ERROR can't use empty string as operand of "+" >> while executing >> "if {([removeLeadingZeros $n] > [removeLeadingZeros $end] + 1)} { >> if {$start == $end} { >> append res "$end," >> } else { >> ..." >> (procedure "groupNumRanges" line 15) >> invoked from within >> "groupNumRanges $NEW_GROUPS($pNs)" >> (procedure "groupingEngine" line 24) >> invoked from within >> "groupingEngine $groups" >> (procedure "compressNames" line 12) >> invoked from within >> "compressNames $mlidHcas" >> (procedure "reportFabQualities" line 82) >> invoked from within >> "reportFabQualities" can't use empty string as operand of "+" >> >> > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general Hi. - ibdiagnet 1.2 crash when encounter a zero sys image guid ==> As you can see, this is fixed in OFED 1.3 - A system image guid is used to identify nodes that belong to the same system. For HCAs, it is purely informational. For switches, it assists the SM in some advances routing features. Bottom line - no "real" harm to the IB functionality if one or more HCAs hasv system image guid 0. - You can set the system image guid using the mstflint tool (Mellanox firmware burning tool). However, if you used tvflash to burn the HCA firmware, it is advised to continue using tvflash (which I'm not familiar with). You can use mstflint to query the device firmware with no risk - Run "mstflint -d mthca0 q" . Regards, Oren. From yosefe at Voltaire.COM Mon Jul 21 07:38:17 2008 From: yosefe at Voltaire.COM (Yossi Etigin) Date: Mon, 21 Jul 2008 17:38:17 +0300 Subject: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned Message-ID: <48849F59.7060502@Voltaire.COM> The problem: A multicast sender joins the MGID as full member, but does not leave (as long as the interface is up). This causes an MGID leakage in the SM. Proposed solution: IPOIB driver will leave an MGID when the kernel neighbour for this MGID is cleaned. Use the ipoib_neigh_cleanup hook. Since it's called from atomic context, schedule a task that will actually leave the group. Allocate a bit IPOIB_MCAST_FLAG_UNUSED that will indicate that a group is marked to be left by this task. An issue we found with this solution, is that it takes a while until the neighbour is cleaned, and it requires that the host will send some traffic (I supposed that unless we put more entries in the routing table, out neighbour will not be garbage collected). I'll be happy to hear remarks/suggestions, particularly about the issue above. The code below probably has issues, it's just to demonstrate the proposed change. -- diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index ca126fc..522e4a0 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -95,6 +95,7 @@ enum { IPOIB_MCAST_FLAG_SENDONLY = 1, IPOIB_MCAST_FLAG_BUSY = 2, /* joining or already joined */ IPOIB_MCAST_FLAG_ATTACHED = 3, + IPOIB_MCAST_FLAG_UNUSED = 4, MAX_SEND_CQE = 16, }; @@ -275,7 +276,8 @@ struct ipoib_dev_priv { struct rb_root multicast_tree; struct delayed_work pkey_poll_task; - struct delayed_work mcast_task; + struct delayed_work mcast_join_task; + struct work_struct mcast_leave_task; struct work_struct flush_task; struct work_struct restart_task; struct delayed_work ah_reap_task; @@ -440,7 +442,9 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port); void ipoib_dev_cleanup(struct net_device *dev); void ipoib_mcast_join_task(struct work_struct *work); +void ipoib_mcast_leave_task(struct work_struct *work); void ipoib_mcast_send(struct net_device *dev, void *mgid, struct sk_buff *skb); +void ipoib_mcast_neigh_cleanup(struct net_device *dev, void *mgid); void ipoib_mcast_restart_task(struct work_struct *work); int ipoib_mcast_start_thread(struct net_device *dev); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 2442090..e299ea2 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -823,6 +823,8 @@ static void ipoib_neigh_cleanup(struct neighbour *n) IPOIB_QPN(n->ha), IPOIB_GID_RAW_ARG(n->ha + 4)); + ipoib_mcast_neigh_cleanup(n->dev, n->ha + 4); + spin_lock_irqsave(&priv->lock, flags); if (neigh->ah) @@ -990,8 +992,9 @@ static void ipoib_setup(struct net_device *dev) INIT_DELAYED_WORK(&priv->pkey_poll_task, ipoib_pkey_poll); INIT_WORK(&priv->pkey_event_task, ipoib_pkey_event); - INIT_DELAYED_WORK(&priv->mcast_task, ipoib_mcast_join_task); - INIT_WORK(&priv->flush_task, ipoib_ib_dev_flush); + INIT_DELAYED_WORK(&priv->mcast_join_task, ipoib_mcast_join_task); + INIT_WORK(&priv->mcast_leave_task, ipoib_mcast_leave_task); + INIT_WORK(&priv->flush_task, ipoib_ib_dev_flush); INIT_WORK(&priv->restart_task, ipoib_mcast_restart_task); INIT_DELAYED_WORK(&priv->ah_reap_task, ipoib_reap_ah); } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 3f663fb..7175479 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -389,7 +389,7 @@ static int ipoib_mcast_join_complete(int status, mutex_lock(&mcast_mutex); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) queue_delayed_work(ipoib_workqueue, - &priv->mcast_task, 0); + &priv->mcast_join_task, 0); mutex_unlock(&mcast_mutex); if (mcast == priv->broadcast) @@ -422,7 +422,7 @@ static int ipoib_mcast_join_complete(int status, mutex_lock(&mcast_mutex); spin_lock_irq(&priv->lock); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) - queue_delayed_work(ipoib_workqueue, &priv->mcast_task, + queue_delayed_work(ipoib_workqueue, &priv->mcast_join_task, mcast->backoff * HZ); spin_unlock_irq(&priv->lock); mutex_unlock(&mcast_mutex); @@ -492,7 +492,7 @@ static void ipoib_mcast_join(struct net_device *dev, struct ipoib_mcast *mcast, mutex_lock(&mcast_mutex); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) queue_delayed_work(ipoib_workqueue, - &priv->mcast_task, + &priv->mcast_join_task, mcast->backoff * HZ); mutex_unlock(&mcast_mutex); } @@ -501,7 +501,7 @@ static void ipoib_mcast_join(struct net_device *dev, struct ipoib_mcast *mcast, void ipoib_mcast_join_task(struct work_struct *work) { struct ipoib_dev_priv *priv = - container_of(work, struct ipoib_dev_priv, mcast_task.work); + container_of(work, struct ipoib_dev_priv, mcast_join_task.work); struct net_device *dev = priv->dev; if (!test_bit(IPOIB_MCAST_RUN, &priv->flags)) @@ -530,7 +530,7 @@ void ipoib_mcast_join_task(struct work_struct *work) mutex_lock(&mcast_mutex); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) queue_delayed_work(ipoib_workqueue, - &priv->mcast_task, HZ); + &priv->mcast_join_task, HZ); mutex_unlock(&mcast_mutex); return; } @@ -591,7 +591,7 @@ int ipoib_mcast_start_thread(struct net_device *dev) mutex_lock(&mcast_mutex); if (!test_and_set_bit(IPOIB_MCAST_RUN, &priv->flags)) - queue_delayed_work(ipoib_workqueue, &priv->mcast_task, 0); + queue_delayed_work(ipoib_workqueue, &priv->mcast_join_task, 0); mutex_unlock(&mcast_mutex); spin_lock_irq(&priv->lock); @@ -613,7 +613,7 @@ int ipoib_mcast_stop_thread(struct net_device *dev, int flush) mutex_lock(&mcast_mutex); clear_bit(IPOIB_MCAST_RUN, &priv->flags); - cancel_delayed_work(&priv->mcast_task); + cancel_delayed_work(&priv->mcast_join_task); mutex_unlock(&mcast_mutex); if (flush) @@ -864,6 +864,46 @@ void ipoib_mcast_restart_task(struct work_struct *work) ipoib_mcast_start_thread(dev); } +void ipoib_mcast_leave_task(struct work_struct *work) +{ + struct ipoib_dev_priv *priv = + container_of(work, struct ipoib_dev_priv, mcast_leave_task); + struct net_device *dev = priv->dev; + struct ipoib_mcast *mcast; + + spin_lock_irq(&priv->lock); + list_for_each_entry(mcast, &priv->multicast_list, list) { + if (test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags) && + test_bit(IPOIB_MCAST_FLAG_UNUSED, &mcast->flags)) { + break; + } + } + spin_unlock_irq(&priv->lock); + + if (&mcast->list == &priv->multicast_list) { + ipoib_dbg(priv, "ipoib_mcast_leave_task: no groups to leave\n"); + return; + } + + list_del(&mcast->list); + rb_erase(&mcast->rb_node, &priv->multicast_tree); + ipoib_mcast_leave(dev, mcast); + ipoib_mcast_free(mcast); +} + +void ipoib_mcast_neigh_cleanup(struct net_device *dev, void *mgid) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct ipoib_mcast *mcast = __ipoib_mcast_find(dev, mgid); + + if (mcast) { + set_bit(IPOIB_MCAST_FLAG_UNUSED, &mcast->flags); + queue_work(ipoib_workqueue, &priv->mcast_leave_task); + } else + ipoib_dbg_mcast(priv, "mcast gid " IPOIB_GID_FMT " not found\n", + IPOIB_GID_RAW_ARG(mgid)); +} + #ifdef CONFIG_INFINIBAND_IPOIB_DEBUG struct ipoib_mcast_iter *ipoib_mcast_iter_init(struct net_device *dev) -- --Yossi From jackm at dev.mellanox.co.il Mon Jul 21 08:06:48 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 21 Jul 2008 18:06:48 +0300 Subject: [ofa-general] [PATCH] mlx4: Add missing FW status return code. Message-ID: <200807211806.49276.jackm@dev.mellanox.co.il> mlx4: Add missing FW status return code. Added ICM_ERROR status code. In mapping to errno's, -ENFILE seems closest. This is in preparation for providing more detailed log info using mlx4_err() in low-level driver when a non-zero status is returned. Signed-off-by: Jack Morgenstein --- drivers/net/mlx4/cmd.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c index 70dff94..04d5bc6 100644 --- a/drivers/net/mlx4/cmd.c +++ b/drivers/net/mlx4/cmd.c @@ -67,6 +67,8 @@ enum { CMD_STAT_BAD_INDEX = 0x0a, /* FW image corrupted: */ CMD_STAT_BAD_NVMEM = 0x0b, + /* Error in ICM mapping (e.g. not enough auxiliary ICM pages to execute command): */ + CMD_STAT_ICM_ERROR = 0x0c, /* Attempt to modify a QP/EE which is not in the presumed state: */ CMD_STAT_BAD_QP_STATE = 0x10, /* Bad segment parameters (Address/Size): */ @@ -119,6 +121,7 @@ static int mlx4_status_to_errno(u8 status) [CMD_STAT_BAD_RES_STATE] = -EBADF, [CMD_STAT_BAD_INDEX] = -EBADF, [CMD_STAT_BAD_NVMEM] = -EFAULT, + [CMD_STAT_ICM_ERROR] = -ENFILE, [CMD_STAT_BAD_QP_STATE] = -EINVAL, [CMD_STAT_BAD_SEG_PARAM] = -EFAULT, [CMD_STAT_REG_BOUND] = -EBUSY, -- 1.5.1.6 From 512jbui163.com at lists.openfabrics.org Mon Jul 21 08:44:22 2008 From: 512jbui163.com at lists.openfabrics.org (=?GB2312?B?1cXPyMn6?=) Date: Mon, 21 Jul 2008 23:44:22 +0800 Subject: [ofa-general] =?gb2312?b?1qQ=?= Message-ID: <20080721154734.5F6E8E60975@openfabrics.org> 你好我公司长期有； 1.熏蒸/消毒证 =120元 10.陶瓷(铅检)证书=120元 2.中华熏蒸/消毒证=120元 11.重量证书=100元 3.飞达熏蒸/消毒证=120元 12.质量证书 =180元 4.亚泰熏蒸/消毒证=120元 13.数量证书=100元 5.庄臣熏蒸/消毒证=120元 14.检验证书--装运前检验证书=220元 6.植物检疫证书 =140元 15.检验证书=160元 7.兽医证书 =120元 16.产地证C/O =130元 F/E=135元 F/F =150元 8.健康证书=100元 17.普惠证F/A=125元 9.卫生证书=120元 如有打扰敬请见谅 联系电话：13642347277 联系人：张先生 联系电话：13410960171 联系人：刘先生 From rdreier at cisco.com Mon Jul 21 09:25:07 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 21 Jul 2008 09:25:07 -0700 Subject: [ofa-general] Re: [PATCH 1/7] drivers/infiniband: Release mutex in error handling code In-Reply-To: (Julia Lawall's message of "Mon, 21 Jul 2008 09:57:04 +0200 (CEST)") References: Message-ID: > --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c > +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c > @@ -163,6 +163,7 @@ static int alloc_small_queue_page(struct > > out: > ehca_err(pd->ib_pd.device, "failed to allocate small queue page"); > + mutex_unlock(&pd->lock); > return 0; > } Thanks, looks obviously correct. ehca guys, I will queue this for my next pull request to Linus; please let me know if you see a problem and want me to drop it. - R. From rdreier at cisco.com Mon Jul 21 09:34:02 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 21 Jul 2008 09:34:02 -0700 Subject: [ofa-general] Re: [PATCH 2/2] IB/ehca: Default value for Local CA ACK Delay In-Reply-To: <200807211316.37224.fenkes@de.ibm.com> (Joachim Fenkes's message of "Mon, 21 Jul 2008 13:16:36 +0200") References: <200807211313.23648.fenkes@de.ibm.com> <200807211316.37224.fenkes@de.ibm.com> Message-ID: thanks, applied 1 and 2 From rdreier at cisco.com Mon Jul 21 09:41:17 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 21 Jul 2008 09:41:17 -0700 Subject: [ofa-general] Re: [PATCH] ib/ipoib : print error code of ib_sa path lookup call In-Reply-To: (Or Gerlitz's message of "Sun, 20 Jul 2008 09:35:38 +0300 (IDT)") References: Message-ID: heh, you managed to get a checkpatch error on the only line you changed: ERROR: space required after that ',' (ctx:VxV) #56: FILE: drivers/infiniband/ulp/ipoib/ipoib_main.c:551: + ipoib_warn(priv, "ib_sa_path_rec_get failed ret=%d\n",path->query_id); ^ anyway, fixed up and applied, thanks. From hal.rosenstock at gmail.com Mon Jul 21 09:46:55 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 21 Jul 2008 12:46:55 -0400 Subject: [ofa-general] [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) In-Reply-To: <200807151745.53152.jackm@dev.mellanox.co.il> References: <200807151745.53152.jackm@dev.mellanox.co.il> Message-ID: On Tue, Jul 15, 2008 at 10:45 AM, Jack Morgenstein wrote: > From 01d8c13b6dd314b239a5b0b76e57320aeaa5c338 Mon Sep 17 00:00:00 2001 > From: Jack Morgenstein > Date: Tue, 15 Jul 2008 17:19:41 +0300 > Subject: [PATCH] mlx4_core: add module parameter to enable/disable QoS support. > > Add a module parameter "enable_qos" to mlx4_core. If this param > is > 0, support for QoS is enabled in the HCA (INIT_HCA command). > By default, the parameter is set to 0 (disabled). Out of curiousity, why is a module parameter needed to enable/disable QoS ? Is there a downside to enabling it in terms of the HCA ? -- Hal > > Signed-off-by: Jack Morgenstein > --- > drivers/net/mlx4/fw.c | 8 ++++++++ > 1 files changed, 8 insertions(+), 0 deletions(-) > > diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c > index 2b5006b..2b60525 100644 > --- a/drivers/net/mlx4/fw.c > +++ b/drivers/net/mlx4/fw.c > @@ -46,6 +46,10 @@ enum { > extern void __buggy_use_of_MLX4_GET(void); > extern void __buggy_use_of_MLX4_PUT(void); > > +static int mlx4_core_enable_qos; > +module_param_named(enable_qos, mlx4_core_enable_qos, int, 0444); > +MODULE_PARM_DESC(enable_qos, "Enable Quality of Service support in the HCA if > 0, (default 0)"); > + > #define MLX4_GET(dest, source, offset) \ > do { \ > void *__p = (char *) (source) + (offset); \ > @@ -737,6 +741,10 @@ int mlx4_INIT_HCA(struct mlx4_dev *dev, struct mlx4_init_hca_param *param) > if (dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) > *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 3); > > + /* Enable QoS support if module parameter set */ > + if (mlx4_core_enable_qos) > + *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 2); > + > /* QPC/EEC/CQC/EQC/RDMARC attributes */ > > MLX4_PUT(inbox, param->qpc_base, INIT_HCA_QPC_BASE_OFFSET); > -- > 1.5.1.6 > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From rdreier at cisco.com Mon Jul 21 09:47:08 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 21 Jul 2008 09:47:08 -0700 Subject: [ofa-general] Re: [PATCH] core/iwcm: remove IB_ACCESS_LOCAL_WRITE in remote QP attributes In-Reply-To: <200807181119.27203.dotanba@gmail.com> (Dotan Barak's message of "Fri, 18 Jul 2008 11:19:26 +0300") References: <200807181119.27203.dotanba@gmail.com> Message-ID: > - qp_attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE | > - IB_ACCESS_REMOTE_WRITE| > + qp_attr->qp_access_flags = IB_ACCESS_REMOTE_WRITE| > IB_ACCESS_REMOTE_READ; makes sense, applied. Steve, this patch is correct, right? This nes code looks really strange: if (attr_mask & IB_QP_ACCESS_FLAGS) { if (attr->qp_access_flags & IB_ACCESS_LOCAL_WRITE) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN | NES_QPCONTEXT_MISC_RDMA_READ_EN); issue_modify_qp = 1; } if (attr->qp_access_flags & IB_ACCESS_REMOTE_WRITE) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN); issue_modify_qp = 1; } if (attr->qp_access_flags & IB_ACCESS_REMOTE_READ) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_READ_EN); issue_modify_qp = 1; } if (attr->qp_access_flags & IB_ACCESS_MW_BIND) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_WBIND_EN); issue_modify_qp = 1; } if (nesqp->user_mode) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN | NES_QPCONTEXT_MISC_RDMA_READ_EN); issue_modify_qp = 1; } } should it just be rewritten to: if (attr->qp_access_flags & IB_ACCESS_REMOTE_WRITE) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN); issue_modify_qp = 1; } if (attr->qp_access_flags & IB_ACCESS_REMOTE_READ) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_READ_EN); issue_modify_qp = 1; } if (attr->qp_access_flags & IB_ACCESS_MW_BIND) { nesqp->nesqp_context->misc |= cpu_to_le32(NES_QPCONTEXT_MISC_WBIND_EN); issue_modify_qp = 1; } I don't see why setting LOCAL_WRITE should imply both RDMA permissions, or why you always want to set those permissions for usermode QPs ?? - R. From swise at opengridcomputing.com Mon Jul 21 10:20:21 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 21 Jul 2008 12:20:21 -0500 Subject: [ofa-general] Re: [PATCH] core/iwcm: remove IB_ACCESS_LOCAL_WRITE in remote QP attributes In-Reply-To: References: <200807181119.27203.dotanba@gmail.com> Message-ID: <4884C555.8070509@opengridcomputing.com> I you go changing these, please test them. I know the Roland Dreier wrote: > > - qp_attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE | > > - IB_ACCESS_REMOTE_WRITE| > > + qp_attr->qp_access_flags = IB_ACCESS_REMOTE_WRITE| > > IB_ACCESS_REMOTE_READ; > > makes sense, applied. Steve, this patch is correct, right? > > It looks correct, but what does it break? :) From ronli at voltaire.com Mon Jul 21 13:27:35 2008 From: ronli at voltaire.com (ronli at voltaire.com) Date: Mon, 21 Jul 2008 20:27:35 +0000 (UTC) Subject: [ofa-general] [PATCH 3/3 v2] ib/uverbs: add support for create_qp_expanded in uverbs Message-ID: This patch adds support for create_qp_expanded to the uverbs. It uses the reserved bitmap in ib_uverbs_create_qp to transfer the new creation flags from the user space to the kernel. Signed-off-by: Ron Livne sq_no_prefetch = 0; - if (init_attr->create_flags & IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) - qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; - if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) qp->flags |= MLX4_IB_QP_LSO; @@ -559,6 +556,10 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } } + if (init_attr->create_flags & + IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) + qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; + err = mlx4_qp_alloc(dev->dev, sqpn, &qp->mqp); if (err) goto err_wrid; @@ -705,8 +706,11 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)) return ERR_PTR(-EINVAL); - if (init_attr->create_flags && - (pd->uobject || init_attr->qp_type != IB_QPT_UD)) + if (init_attr->create_flags && init_attr->qp_type != IB_QPT_UD) + return ERR_PTR(-EINVAL); + + if ((init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) && + pd->uobject) return ERR_PTR(-EINVAL); switch (init_attr->qp_type) { @@ -722,6 +726,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, return ERR_PTR(-ENOMEM); err = create_qp_common(dev, pd, init_attr, udata, 0, qp); + if (err) { kfree(qp); return ERR_PTR(err); diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h index 0df90d8..c744bca 100644 --- a/include/rdma/ib_user_verbs.h +++ b/include/rdma/ib_user_verbs.h @@ -90,6 +90,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, IB_USER_VERBS_CMD_REG_XRC_RCV_QP, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED, }; /* From ronli at voltaire.com Mon Jul 21 13:25:47 2008 From: ronli at voltaire.com (ronli at voltaire.com) Date: Mon, 21 Jul 2008 20:25:47 +0000 (UTC) Subject: [ofa-general] ***SPAM*** [PATCH 1/3 v2] libibverbs : add new verbs: create_qp_expanded Message-ID: 1. This patch adds a new capability flags to the enum ibv_device_cap_flags IBV_DEVICE_BLOCK_MULTICAST_LOOPBACK. which implies that the device is capable of blocking multicast loopback packets. 2. This patch also adds a new verb to the libibverbs: struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr, uint32_t create_flags); which works similarly to ibv_create_qp, except for that it now takes another argument: uint32_t create_flags these creation flags should be aligned with those in the ib_verbs.h in the kernel. 3. New creation flags added: IBV_QP_CREATE_IPOIB_UD_LSO, IBV_MULTICAST_LOOPBACK_BLOCK Changes in v2: LSO creation flag deleted from user space. Added struct ibv_create_qp_expanded. Signed-off-by: Ron Livne diff --git a/include/infiniband/driver.h b/include/infiniband/driver.h index f8138ef..ea824c8 100644 --- a/include/infiniband/driver.h +++ b/include/infiniband/driver.h @@ -155,6 +155,11 @@ int ibv_cmd_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int ibv_cmd_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + uint32_t create_flags, + struct ibv_create_qp_expanded *cmd, size_t cmd_size, + struct ibv_create_qp_resp *resp, size_t resp_size); /* * sysfs helper functions diff --git a/include/infiniband/kern-abi.h b/include/infiniband/kern-abi.h index 8b5cd9a..7185709 100644 --- a/include/infiniband/kern-abi.h +++ b/include/infiniband/kern-abi.h @@ -94,6 +94,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, IB_USER_VERBS_CMD_REG_XRC_RCV_QP, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED, }; /* @@ -459,6 +460,28 @@ struct ibv_create_qp { __u64 driver_data[0]; }; +struct ibv_create_qp_expanded { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u64 user_handle; + __u32 pd_handle; + __u32 send_cq_handle; + __u32 recv_cq_handle; + __u32 srq_handle; + __u32 max_send_wr; + __u32 max_recv_wr; + __u32 max_send_sge; + __u32 max_recv_sge; + __u32 max_inline_data; + __u8 sq_sig_all; + __u8 qp_type; + __u8 is_srq; + __u8 create_flags; + __u64 driver_data[0]; +}; + struct ibv_create_qp_resp { __u32 qp_handle; __u32 qpn; @@ -945,6 +968,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP_V2 = -1, IB_USER_VERBS_CMD_REG_XRC_RCV_QP_V2 = -1, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED_V2 = -1, }; struct ibv_destroy_cq_v1 { diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index 2865d5c..7bc0d18 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -93,7 +93,8 @@ enum ibv_device_cap_flags { IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, IBV_DEVICE_SRQ_RESIZE = 1 << 13, IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, - IBV_DEVICE_XRC = 1 << 20 + IBV_DEVICE_XRC = 1 << 20, + IBV_DEVICE_BLOCK_MULTICAST_LOOPBACK = 1 << 22 }; enum ibv_atomic_cap { @@ -631,6 +632,16 @@ enum { IBV_SYSFS_PATH_MAX = 256 }; +/* + This enum must be alligned with ib_qp_create_flags + in include/rdma/ib_verbs.h +*/ +enum ibv_qp_create_flags { + IBV_QP_CREATE_MULTICAST_LOOPBACK_BLOCK = 1 << 1 +}; + + + struct ibv_device { struct ibv_device_ops ops; enum ibv_node_type node_type; @@ -668,6 +679,9 @@ struct ibv_more_ops { uint32_t xrc_qp_num); int (*unreg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); + struct ibv_qp * (*create_qp_expanded)(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags); }; @@ -1029,6 +1043,13 @@ struct ibv_qp *ibv_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); /** + * ibv_create_qp_expanded - Create a queue pair with creation flags. + */ +struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags); + +/** * ibv_modify_qp - Modify a queue pair. */ int ibv_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, diff --git a/src/cmd.c b/src/cmd.c index cedf55e..0bf908d 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -625,8 +625,18 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, struct ibv_create_qp *cmd, size_t cmd_size, struct ibv_create_qp_resp *resp, size_t resp_size) { - IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP, resp, resp_size); + return ibv_cmd_create_qp_expanded(pd, qp, attr, 0, cmd, cmd_size, resp, resp_size); +} + +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + uint32_t create_flags, + struct ibv_create_qp_expanded *cmd, size_t cmd_size, + struct ibv_create_qp_resp *resp, size_t resp_size) +{ + IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP_EXPANDED, resp, resp_size); + cmd->create_flags = create_flags; cmd->user_handle = (uintptr_t) qp; cmd->pd_handle = pd->handle; cmd->send_cq_handle = attr->send_cq->handle; @@ -642,7 +652,6 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? (attr->xrc_domain ? attr->xrc_domain->handle : 0) : (attr->srq ? attr->srq->handle : 0); - cmd->reserved = 0; if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; diff --git a/src/libibverbs.map b/src/libibverbs.map index fce6965..a042e31 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -112,4 +112,7 @@ IBVERBS_1.1 { ibv_port_state_str; ibv_event_type_str; ibv_wc_status_str; + + ibv_create_qp_expanded; + ibv_cmd_create_qp_expanded; } IBVERBS_1.0; diff --git a/src/verbs.c b/src/verbs.c index 4bfeec2..ae8dd7e 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -449,6 +449,31 @@ struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, } default_symver(__ibv_create_qp, ibv_create_qp); +struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags) +{ + struct ibv_qp *qp = pd->context->more_ops->create_qp_expanded(pd, qp_init_attr, create_flags); + + if (qp) { + qp->context = pd->context; + qp->qp_context = qp_init_attr->qp_context; + qp->pd = pd; + qp->send_cq = qp_init_attr->send_cq; + qp->recv_cq = qp_init_attr->recv_cq; + qp->srq = qp_init_attr->srq; + qp->qp_type = qp_init_attr->qp_type; + qp->state = IBV_QPS_RESET; + qp->events_completed = 0; + qp->xrc_domain = qp_init_attr->qp_type == IBV_QPT_XRC ? + qp_init_attr->xrc_domain : NULL; + pthread_mutex_init(&qp->mutex, NULL); + pthread_cond_init(&qp->cond, NULL); + } + + return qp; +} + int __ibv_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, enum ibv_qp_attr_mask attr_mask, struct ibv_qp_init_attr *init_attr) @@ -610,7 +635,7 @@ int ibv_create_xrc_rcv_qp(struct ibv_qp_init_attr *init_attr, return ENOSYS; return c->more_ops->create_xrc_rcv_qp(init_attr, - xrc_rcv_qpn); + xrc_rcv_qpn); } int ibv_modify_xrc_rcv_qp(struct ibv_xrc_domain *d, From ronli at voltaire.com Mon Jul 21 13:26:43 2008 From: ronli at voltaire.com (ronli at voltaire.com) Date: Mon, 21 Jul 2008 20:26:43 +0000 (UTC) Subject: [ofa-general] ***SPAM*** [PATCH 2/3 v2] libmlx4: add support for create_qp_expanded Message-ID: Adds support for the new verb: create_qp_expanded in libmlx4. Changes in v2: Minimized code duplications by calling mlx4_create_qp_expanded from mlx4_create_qp. Signed-off-by: Ron Livne diff --git a/src/mlx4.c b/src/mlx4.c index 1f23144..bfcb7ec 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -79,6 +79,7 @@ static struct ibv_more_ops mlx4_more_ops = { .query_xrc_rcv_qp = mlx4_query_xrc_rcv_qp, .reg_xrc_rcv_qp = mlx4_reg_xrc_rcv_qp, .unreg_xrc_rcv_qp = mlx4_unreg_xrc_rcv_qp, + .create_qp_expanded = mlx4_create_qp_expanded, #endif }; #endif diff --git a/src/mlx4.h b/src/mlx4.h index a2e3aec..7ed3ac8 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -427,6 +427,9 @@ int mlx4_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags); #endif diff --git a/src/verbs.c b/src/verbs.c index b1bad3c..c50e455 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -404,110 +404,7 @@ int mlx4_destroy_srq(struct ibv_srq *ibsrq) struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) { - struct mlx4_create_qp cmd; - struct ibv_create_qp_resp resp; - struct mlx4_qp *qp; - int ret; - - /* Sanity check QP size before proceeding */ - if (attr->cap.max_send_wr > 65536 || - attr->cap.max_recv_wr > 65536 || - attr->cap.max_send_sge > 64 || - attr->cap.max_recv_sge > 64 || - attr->cap.max_inline_data > 1024) - return NULL; - - qp = malloc(sizeof *qp); - if (!qp) - return NULL; - - mlx4_calc_sq_wqe_size(&attr->cap, attr->qp_type, qp); - - /* - * We need to leave 2 KB + 1 WQE of headroom in the SQ to - * allow HW to prefetch. - */ - qp->sq_spare_wqes = (2048 >> qp->sq.wqe_shift) + 1; - qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); - qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); - - if (attr->srq || attr->qp_type == IBV_QPT_XRC) - attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; - else { - if (attr->cap.max_recv_sge < 1) - attr->cap.max_recv_sge = 1; - if (attr->cap.max_recv_wr < 1) - attr->cap.max_recv_wr = 1; - } - - if (mlx4_alloc_qp_buf(pd, &attr->cap, attr->qp_type, qp)) - goto err; - - mlx4_init_qp_indices(qp); - - if (pthread_spin_init(&qp->sq.lock, PTHREAD_PROCESS_PRIVATE) || - pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) - goto err_free; - - if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { - qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); - if (!qp->db) - goto err_free; - - *qp->db = 0; - } - - cmd.buf_addr = (uintptr_t) qp->buf.buf; - if (attr->srq || attr->qp_type == IBV_QPT_XRC) - cmd.db_addr = 0; - else - cmd.db_addr = (uintptr_t) qp->db; - cmd.log_sq_stride = qp->sq.wqe_shift; - for (cmd.log_sq_bb_count = 0; - qp->sq.wqe_cnt > 1 << cmd.log_sq_bb_count; - ++cmd.log_sq_bb_count) - ; /* nothing */ - cmd.sq_no_prefetch = 0; /* OK for ABI 2: just a reserved field */ - memset(cmd.reserved, 0, sizeof cmd.reserved); - - ret = ibv_cmd_create_qp(pd, &qp->ibv_qp, attr, &cmd.ibv_cmd, sizeof cmd, - &resp, sizeof resp); - if (ret) - goto err_rq_db; - - ret = mlx4_store_qp(to_mctx(pd->context), qp->ibv_qp.qp_num, qp); - if (ret) - goto err_destroy; - - qp->rq.wqe_cnt = qp->rq.max_post = attr->cap.max_recv_wr; - qp->rq.max_gs = attr->cap.max_recv_sge; - mlx4_set_sq_sizes(qp, &attr->cap, attr->qp_type); - - qp->doorbell_qpn = htonl(qp->ibv_qp.qp_num << 8); - if (attr->sq_sig_all) - qp->sq_signal_bits = htonl(MLX4_WQE_CTRL_CQ_UPDATE); - else - qp->sq_signal_bits = 0; - - return &qp->ibv_qp; - -err_destroy: - ibv_cmd_destroy_qp(&qp->ibv_qp); - -err_rq_db: - if (!attr->srq && attr->qp_type != IBV_QPT_XRC) - mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); - -err_free: - free(qp->sq.wrid); - if (qp->rq.wqe_cnt) - free(qp->rq.wrid); - mlx4_free_buf(&qp->buf); - -err: - free(qp); - - return NULL; + return mlx4_create_qp_expanded(pd, attr, 0); } int mlx4_query_qp(struct ibv_qp *ibqp, struct ibv_qp_attr *attr, @@ -825,4 +722,115 @@ int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, return ibv_cmd_unreg_xrc_rcv_qp(xrc_domain, xrc_qp_num); } +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags) +{ + struct mlx4_create_qp cmd; + struct ibv_create_qp_resp resp; + struct mlx4_qp *qp; + int ret; + + /* Sanity check QP size before proceeding */ + if (attr->cap.max_send_wr > 65536 || + attr->cap.max_recv_wr > 65536 || + attr->cap.max_send_sge > 64 || + attr->cap.max_recv_sge > 64 || + attr->cap.max_inline_data > 1024) + return NULL; + + qp = malloc(sizeof *qp); + if (!qp) + return NULL; + + mlx4_calc_sq_wqe_size(&attr->cap, attr->qp_type, qp); + + /* + * We need to leave 2 KB + 1 WQE of headroom in the SQ to + * allow HW to prefetch. + */ + qp->sq_spare_wqes = (2048 >> qp->sq.wqe_shift) + 1; + qp->sq.wqe_cnt = align_queue_size(attr->cap.max_send_wr + qp->sq_spare_wqes); + qp->rq.wqe_cnt = align_queue_size(attr->cap.max_recv_wr); + + if (attr->srq || attr->qp_type == IBV_QPT_XRC) + attr->cap.max_recv_wr = qp->rq.wqe_cnt = 0; + else { + if (attr->cap.max_recv_sge < 1) + attr->cap.max_recv_sge = 1; + if (attr->cap.max_recv_wr < 1) + attr->cap.max_recv_wr = 1; + } + + if (mlx4_alloc_qp_buf(pd, &attr->cap, attr->qp_type, qp)) + goto err; + + mlx4_init_qp_indices(qp); + + if (pthread_spin_init(&qp->sq.lock, PTHREAD_PROCESS_PRIVATE) || + pthread_spin_init(&qp->rq.lock, PTHREAD_PROCESS_PRIVATE)) + goto err_free; + + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) { + qp->db = mlx4_alloc_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ); + if (!qp->db) + goto err_free; + + *qp->db = 0; + } + + cmd.buf_addr = (uintptr_t) qp->buf.buf; + if (attr->srq || attr->qp_type == IBV_QPT_XRC) + cmd.db_addr = 0; + else + cmd.db_addr = (uintptr_t) qp->db; + cmd.log_sq_stride = qp->sq.wqe_shift; + for (cmd.log_sq_bb_count = 0; + qp->sq.wqe_cnt > 1 << cmd.log_sq_bb_count; + ++cmd.log_sq_bb_count) + ; /* nothing */ + cmd.sq_no_prefetch = 0; + + ret = ibv_cmd_create_qp_expanded(pd, &qp->ibv_qp, attr, create_flags, + &cmd.ibv_cmd, sizeof cmd, + &resp, sizeof resp); + if (ret) + goto err_rq_db; + + ret = mlx4_store_qp(to_mctx(pd->context), qp->ibv_qp.qp_num, qp); + if (ret) + goto err_destroy; + + qp->rq.wqe_cnt = qp->rq.max_post = attr->cap.max_recv_wr; + qp->rq.max_gs = attr->cap.max_recv_sge; + mlx4_set_sq_sizes(qp, &attr->cap, attr->qp_type); + + qp->doorbell_qpn = htonl(qp->ibv_qp.qp_num << 8); + if (attr->sq_sig_all) + qp->sq_signal_bits = htonl(MLX4_WQE_CTRL_CQ_UPDATE); + else + qp->sq_signal_bits = 0; + + return &qp->ibv_qp; + +err_destroy: + ibv_cmd_destroy_qp(&qp->ibv_qp); + +err_rq_db: + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); + +err_free: + free(qp->sq.wrid); + if (qp->rq.wqe_cnt) + free(qp->rq.wrid); + mlx4_free_buf(&qp->buf); + +err: + free(qp); + + return NULL; +} + + #endif From ronli at voltaire.com Mon Jul 21 13:28:47 2008 From: ronli at voltaire.com (ronli at voltaire.com) Date: Mon, 21 Jul 2008 20:28:47 +0000 (UTC) Subject: [ofa-general] ***SPAM*** [PATCH 1/1] librdmacm: add support for create qp expanded Message-ID: Adds a new function: int rdma_create_qp_expanded which uses the ibv_create_qp_expanded function in libibverbs and uses it similarly to ibv_create_qp, with the difference of creation flags. Signed-off-by: Ron Livne diff --git a/include/rdma/rdma_cma.h b/include/rdma/rdma_cma.h index a516ab8..34c6b9f 100644 --- a/include/rdma/rdma_cma.h +++ b/include/rdma/rdma_cma.h @@ -296,6 +296,10 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms); int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); +int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags); + /** * rdma_destroy_qp - Deallocate a QP. * @id: RDMA identifier. diff --git a/src/cma.c b/src/cma.c index ecb41bc..92064c8 100644 --- a/src/cma.c +++ b/src/cma.c @@ -786,6 +786,13 @@ static int ucma_init_ud_qp(struct cma_id_private *id_priv, struct ibv_qp *qp) int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr) { + return rdma_create_qp_expanded(id, pd, qp_init_attr, 0); +} + +int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags) +{ struct cma_id_private *id_priv; struct ibv_qp *qp; int ret; @@ -794,7 +801,7 @@ int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, if (id->verbs != pd->context) return -EINVAL; - qp = ibv_create_qp(pd, qp_init_attr); + qp = ibv_create_qp_expanded(pd, qp_init_attr, create_flags); if (!qp) return -ENOMEM; diff --git a/src/librdmacm.map b/src/librdmacm.map index cb94efe..b237eda 100644 --- a/src/librdmacm.map +++ b/src/librdmacm.map @@ -28,5 +28,6 @@ RDMACM_1.0 { rdma_get_local_addr; rdma_get_peer_addr; rdma_migrate_id; + rdma_create_qp_expanded; local: *; }; From HNGUYEN at de.ibm.com Mon Jul 21 11:28:59 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Mon, 21 Jul 2008 20:28:59 +0200 Subject: [ofa-general] Re: [PATCH 1/7] drivers/infiniband: Release mutex in error handling code In-Reply-To: Message-ID: Roland and Julia, Looks good to me. Thanks for pointing this out. Nam Roland Dreier wrote on 21.07.2008 18:25:07: > > --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c > > +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c > > @@ -163,6 +163,7 @@ static int alloc_small_queue_page(struct > > > > out: > > ehca_err(pd->ib_pd.device, "failed to allocate small queue page"); > > + mutex_unlock(&pd->lock); > > return 0; > > } > > Thanks, looks obviously correct. ehca guys, I will queue this for my > next pull request to Linus; please let me know if you see a problem and > want me to drop it. > > - R. From sean.hefty at intel.com Mon Jul 21 11:30:24 2008 From: sean.hefty at intel.com (Hefty, Sean) Date: Mon, 21 Jul 2008 11:30:24 -0700 Subject: [ofa-general] IBCM and ordering of OpenFabrics devices In-Reply-To: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> Message-ID: >1. Technically, mlx4_0 is the first IB device. Should it therefore be >using ucm0? I.e., is libibcm wrong for trying to use ucm1? (note that >OMPI's openib BTL is currently replicating the logic from libibcm to >check for the Right ucm* file so that we can silently fail before >ib_cm_open_device() fails with a warning message -- so if libibcm's >logic to find the Right ucm* file changes, we'll also need to change >MPI's logic to mirror it. OMPI's logic becomes moot in newer libibcm >versions where Sean removed the warning message, though). I think what may be happening is that uverbs1 should map to ucm0, but the library opens ucm1 instead. And uverbs2 should map to ucm1, but an attempt is made to open ucm2, which causes the failure that you're seeing. The libibcm needs to determine the correct file to open using something other than just the uverbs name. That was just a convenient way to get the correct file name, but was never updated once libibverbs supported non-IB devices. - Sean From hal.rosenstock at gmail.com Mon Jul 21 11:58:07 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 21 Jul 2008 14:58:07 -0400 Subject: [ofa-general] reloading/applying configuration changes by opensm In-Reply-To: References: Message-ID: On Sun, Jul 20, 2008 at 6:43 AM, Or Gerlitz wrote: > For some reason... my thinking was that each time opensm does heavy sweep > it reads the cache file and if something was changed there from the previous > sweep, apply this change (eg LMC was 0 and is now 1), is it correct? > > Practically, I see that it doesn't work this way, i.e I run opensm, then > sends it a SIGHUP which indeed seems to cause a heavy sweep as noted in > the man page, but the LMC change I made in the cache file doesn't apply by > the sm to the fabric. > > Maybe this is new feature for which I need to use the master code ? Sasha has made some changes in this area to make things more dynamic in terms of rereading all the various config files but this work is in progress AFAIK. LMC might work on the master or even that latest tar ball version (OpenSM 3.2.2). -- Hal > > Or. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From grthyji126.com at lists.openfabrics.org Mon Jul 21 12:02:40 2008 From: grthyji126.com at lists.openfabrics.org (=?GB2312?B?1cXPyMn6?=) Date: Tue, 22 Jul 2008 03:02:40 +0800 Subject: [ofa-general] =?gb2312?b?MLeiMMaxMA==?= Message-ID: <20080721190552.B8FD5E609CD@openfabrics.org> 你好 我公司长期有商品销售 建筑安装 广告 服务 运输 餐饮定额等 发票代理 ， 如需请联系， 联系电话：13642347225 联系人：张先生 如有打扰，敬请见谅 ！ From jsquyres at cisco.com Mon Jul 21 12:03:16 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Mon, 21 Jul 2008 15:03:16 -0400 Subject: [ofa-general] IBCM and ordering of OpenFabrics devices In-Reply-To: References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> Message-ID: On Jul 21, 2008, at 2:30 PM, Hefty, Sean wrote: >> 1. Technically, mlx4_0 is the first IB device. Should it therefore be >> using ucm0? I.e., is libibcm wrong for trying to use ucm1? (note that >> OMPI's openib BTL is currently replicating the logic from libibcm to >> check for the Right ucm* file so that we can silently fail before >> ib_cm_open_device() fails with a warning message -- so if libibcm's >> logic to find the Right ucm* file changes, we'll also need to change >> MPI's logic to mirror it. OMPI's logic becomes moot in newer libibcm >> versions where Sean removed the warning message, though). > > I think what may be happening is that uverbs1 should map to ucm0, but > the library opens ucm1 instead. And uverbs2 should map to ucm1, but > an > attempt is made to open ucm2, which causes the failure that you're > seeing. The libibcm needs to determine the correct file to open using > something other than just the uverbs name. > > That was just a convenient way to get the correct file name, but was > never updated once libibverbs supported non-IB devices. Gotcha. Do you have a guesstimate on when you could have a patch available? -- Jeff Squyres Cisco Systems From rflores at univaud.com Mon Jul 21 13:16:09 2008 From: rflores at univaud.com (Roderick Flores) Date: Mon, 21 Jul 2008 14:16:09 -0600 Subject: [ofa-general] dhclient will not accept offer Message-ID: <0E95C6BD-672F-4AFA-B651-8A6506EB3DC9@univaud.com> All: I have two test machines, ib0-test and ib1-test, connect together by a pair of QLogic InfiniPath QLE7140 HCAs. There is no router involved. My goal is to test DHCP provisioning onver infiniband on this simple topology. Unfortunately I am unable to get the client to accept the offered address. I have the DHCP daemon running on ib0-test. I have given the ib0 interface the following configuration: DEVICE=ib0 ONBOOT=yes BOOTPROTO=static IPADDR=172.10.10.2 NETMASK=255.255.255.0 If I set the ib0 interface on ib1-test to a static IP similar to this one, everything works well. However, if I try DHCP, it will not accept the offered IP address. This is the configuration on ib1-test: DEVICE=ib0 ONBOOT=yes BOOTPROTO=dhcp If I do a tcpdump on both nodes during the DHCP request, I get very similar answers. On the dhcpd host, ib0-test, I get: 18:40:32.982491 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:40:32.984083 arp who-has 172.10.10.158 tell 172.10.10.2 hardware #32 18:40:33.001142 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:40:33.984028 arp who-has 172.10.10.158 tell 172.10.10.2 hardware #32 18:40:34.983974 arp who-has 172.10.10.158 tell 172.10.10.2 hardware #32 18:40:37.991935 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:40:37.992125 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:40:52.992217 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:40:52.992428 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:41:13.991209 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:41:13.991384 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:41:23.991705 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:41:23.991909 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 Whereas on the client host, ib1-test, I get: 18:40:32.991272 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:40:32.992963 arp who-has 172.10.10.158 tell 172.10.10.2 hardware #32 18:40:33.010013 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:40:33.992895 arp who-has 172.10.10.158 tell 172.10.10.2 hardware #32 18:40:34.992846 arp who-has 172.10.10.158 tell 172.10.10.2 hardware #32 18:40:38.000761 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:40:38.001014 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:40:53.001075 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:40:53.001363 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:41:14.000117 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:41:14.000366 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 18:41:24.000642 IP ib1-test.bootpc > 255.255.255.255.bootps: BOOTP/ DHCP, Request, length: 300 18:41:24.000917 IP 172.10.10.2.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length: 309 There are no substantive differences between these two transfers. Having said, that, there are differences at the packet level. I will not insert all of the packet data for brevity's sake. However I can offer an example: one packet may have the following data: < 0000 00 00 00 20 00 00 00 00 00 00 00 00 00 00 08 00 > 0000 00 04 00 20 00 00 00 00 00 00 00 00 00 00 08 00 while a subsequent packet may have: < 0000 00 04 00 20 00 00 00 00 00 00 00 00 00 00 08 06 > 0000 00 00 00 20 00 00 00 00 00 00 00 00 00 00 08 06 I will not claim to understand enough about TCP/IP transport to say whether this is normal. From a pattern perspective, it appears to be in line with what I would expect. The TCP dumps are in line with the output I get in /var/log/ messages. From the dhcpd host machine I get: Jul 18 17:41:35 ib0-test dhcpd: DHCPDISCOVER from via ib0 Jul 18 17:41:36 ib0-test dhcpd: DHCPOFFER on 172.10.10.158 to via ib0 Jul 18 17:41:40 ib0-test dhcpd: DHCPDISCOVER from via ib0 Jul 18 17:41:40 ib0-test dhcpd: DHCPOFFER on 172.10.10.158 to via ib0 Jul 18 17:41:54 ib0-test dhcpd: DHCPDISCOVER from via ib0 Jul 18 17:41:54 ib0-test dhcpd: DHCPOFFER on 172.10.10.158 to via ib0 Jul 18 17:42:13 ib0-test dhcpd: DHCPDISCOVER from via ib0 Jul 18 17:42:13 ib0-test dhcpd: DHCPOFFER on 172.10.10.158 to via ib0 Jul 18 17:42:23 ib0-test dhcpd: DHCPDISCOVER from via ib0 Jul 18 17:42:23 ib0-test dhcpd: DHCPOFFER on 172.10.10.158 to via ib0 while the client machine has: Jul 18 17:41:35 ib1-test dhclient: Sending on Socket/fallback Jul 18 17:41:35 ib1-test dhclient: DHCPDISCOVER on ib0 to 255.255.255.255 port 67 interval 6 Jul 18 17:41:41 ib1-test dhclient: DHCPDISCOVER on ib0 to 255.255.255.255 port 67 interval 14 Jul 18 17:41:54 ib1-test dhclient: DHCPDISCOVER on ib0 to 255.255.255.255 port 67 interval 19 Jul 18 17:42:14 ib1-test dhclient: DHCPDISCOVER on ib0 to 255.255.255.255 port 67 interval 10 Jul 18 17:42:24 ib1-test dhclient: DHCPDISCOVER on ib0 to 255.255.255.255 port 67 interval 12 Jul 18 17:42:35 ib1-test dhclient: No DHCPOFFERS received. What baffles me is that the offer is made but never accepted. I have tried any number of changes to the dhclient.conf file to avoid rejections, to no avail. Finally, I will leave you with the relevant configuration files... /etc/dhcpd.conf on ib0-test: ddns-update-style interim; ignore client-updates; subnet 172.10.10.0 netmask 255.255.255.0 { range dynamic-bootp 172.10.10.100 172.10.10.200; option domain-name "univaud.com"; option domain-name-servers 192.168.31.10, 10.10.0.12, 10.10.0.13; #option routers 192.168.31.1; option subnet-mask 255.255.255.0; option time-offset -21600; # Central Standard Time option ntp-servers 64.113.32.5, 65.111.164.223, 72.52.190.26; default-lease-time 21600; max-lease-time 43200; } /etc/dhclient.conf on ib1-test: interface "ib0" { send dhcp-client-identifier 20:00:55:00:01:FE: 80:00:00:00:00:00:00:00:11:75:00:00:ff:94:fd; } I created the client identifier from the following information: 20:<4 byte QP Number><8-byte subnet prefix><8 byte GUID>. This seems to be in line with the patch that we applied to the DHCP software to make it work with infiniband. Lastly, an excerpt from /etc/modules.conf on ip1-test: alias ib0 ib_ipoib I tried to use ipath_ether but qlogic's website only has the RHEL4 release (despite having a RHEL5 label). Cheers!!! -- Roderick Flores Solutions Architect Univa UD rflores at univaud.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.hefty at intel.com Mon Jul 21 14:01:11 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 21 Jul 2008 14:01:11 -0700 Subject: [ofa-general] IBCM and ordering of OpenFabrics devices In-Reply-To: References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> Message-ID: <000201c8eb74$e7b40ed0$64d8180a@amr.corp.intel.com> >Do you have a guesstimate on when you could have a patch available? I will try to get something out by the end of the week. - Sean From jsquyres at cisco.com Mon Jul 21 14:08:13 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Mon, 21 Jul 2008 17:08:13 -0400 Subject: [ofa-general] IBCM and ordering of OpenFabrics devices In-Reply-To: <000201c8eb74$e7b40ed0$64d8180a@amr.corp.intel.com> References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> <000201c8eb74$e7b40ed0$64d8180a@amr.corp.intel.com> Message-ID: On Jul 21, 2008, at 5:01 PM, Sean Hefty wrote: >> Do you have a guesstimate on when you could have a patch available? > > I will try to get something out by the end of the week. Many thanks, Sean. I can give it a whirl on my systems if you need some testing. -- Jeff Squyres Cisco Systems From ctung at NetEffect.com Mon Jul 21 15:37:42 2008 From: ctung at NetEffect.com (Chien Tung) Date: Mon, 21 Jul 2008 17:37:42 -0500 Subject: [ofa-general] RE: [PATCH] core/iwcm: remove IB_ACCESS_LOCAL_WRITE in remote QP attributes In-Reply-To: References: <200807181119.27203.dotanba@gmail.com> Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC08311F48@venom2> > From: Roland Dreier [mailto:rdreier at cisco.com] > This nes code looks really strange: > > if (attr_mask & IB_QP_ACCESS_FLAGS) { > if (attr->qp_access_flags & IB_ACCESS_LOCAL_WRITE) { > nesqp->nesqp_context->misc |= > cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN | > > NES_QPCONTEXT_MISC_RDMA_READ_EN); > issue_modify_qp = 1; > } This one maybe a workaround for a silicon bug. I have to look into it to be sure. > if (nesqp->user_mode) { > nesqp->nesqp_context->misc |= > cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN | > > NES_QPCONTEXT_MISC_RDMA_READ_EN); > issue_modify_qp = 1; > } > } > I don't see why setting LOCAL_WRITE should imply both RDMA > permissions, or why you always want to set those permissions > for usermode QPs ?? There is a comment in our CVS source indicating that was a workaround for a CM bug in OFED 1.2. We may be able to remove this one. Chien From dave.olson at qlogic.com Mon Jul 21 19:16:50 2008 From: dave.olson at qlogic.com (Dave Olson) Date: Mon, 21 Jul 2008 19:16:50 -0700 (PDT) Subject: [ofa-general] dhclient will not accept offer In-Reply-To: <0E95C6BD-672F-4AFA-B651-8A6506EB3DC9@univaud.com> References: <0E95C6BD-672F-4AFA-B651-8A6506EB3DC9@univaud.com> Message-ID: On Mon, 21 Jul 2008, Roderick Flores wrote: | I created the client identifier from the following information: | | 20:<4 byte QP Number><8-byte subnet prefix><8 byte GUID>. | | This seems to be in line with the patch that we applied to the DHCP software | to make it work with infiniband. I've not been able to get dhcp to work with ipoib, but I've not tried that patch. Has anybody gotten it to work with recent OFED on any IB HCAs? | Lastly, an excerpt from /etc/modules.conf on ip1-test: | | alias ib0 ib_ipoib | | I tried to use ipath_ether but qlogic's website only has the RHEL4 release | (despite having a RHEL5 label). It's the same set of rpms for RHEL4 and RHEL5 (identical tarball). The marketing setup insists on a per-distro download page, but we don't do independent builds and testing, just the one RHEL build (which also covers CentOS and Scientific Linux). Dave Olson dave.olson at qlogic.com From dpn at isomerica.net Mon Jul 21 20:07:07 2008 From: dpn at isomerica.net (=?UTF-8?B?RGFuIE5vw6k=?=) Date: Mon, 21 Jul 2008 23:07:07 -0400 Subject: [ofa-general] Latency of IB vs. iWARP (10GE) Message-ID: <48854EDB.8080904@isomerica.net> My understanding is that the latency of InfiniBand is lower than 10GE iWARP. I'm particular concerned about smaller messages. I have not been able to find any solid benchmarks (prefer a memory to memory measurement) and unfortunately I don't currently have any hardware to evaluate this. Does anyone have any benchmark results or other insight they could share? Thanks. Cheers, Dan -- /--------------- - - - - - - | Dan Noé | http://isomerica.net/~dpn/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 257 bytes Desc: OpenPGP digital signature URL: From rdreier at cisco.com Mon Jul 21 22:21:16 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 21 Jul 2008 22:21:16 -0700 Subject: [ofa-general] Re: [PATCH 1/3 v2] libibverbs : add new verbs: create_qp_expanded In-Reply-To: (ronli@voltaire.com's message of "Mon, 21 Jul 2008 20:25:47 +0000 (UTC)") References: Message-ID: > --- a/src/cmd.c > +++ b/src/cmd.c > @@ -625,8 +625,18 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, > struct ibv_create_qp *cmd, size_t cmd_size, > struct ibv_create_qp_resp *resp, size_t resp_size) > { > - IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP, resp, resp_size); > + return ibv_cmd_create_qp_expanded(pd, qp, attr, 0, cmd, cmd_size, resp, resp_size); > +} Guys, please try to be careful about compatibility... we care about both new-kernel/old-libibverbs and old-kernel/new-libibverbs. In this case think about what happens if we merge this into libibverbs and then a user tries that code on an old kernel. - R. From rdreier at cisco.com Mon Jul 21 22:25:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 21 Jul 2008 22:25:05 -0700 Subject: [ofa-general] RE: [PATCH] core/iwcm: remove IB_ACCESS_LOCAL_WRITE in remote QP attributes In-Reply-To: <5E701717F2B2ED4EA60F87C8AA57B7CC08311F48@venom2> (Chien Tung's message of "Mon, 21 Jul 2008 17:37:42 -0500") References: <200807181119.27203.dotanba@gmail.com> <5E701717F2B2ED4EA60F87C8AA57B7CC08311F48@venom2> Message-ID: > > if (attr_mask & IB_QP_ACCESS_FLAGS) { > > if (attr->qp_access_flags & IB_ACCESS_LOCAL_WRITE) { > > nesqp->nesqp_context->misc |= > > cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN | > > > > NES_QPCONTEXT_MISC_RDMA_READ_EN); > > issue_modify_qp = 1; > > } > > This one maybe a workaround for a silicon bug. I have to look > into it to be sure. not sure I see how this could be a HW bug workaround... this is setting some QP flags based on some random flag that a user shouldn't be setting. > > if (nesqp->user_mode) { > > nesqp->nesqp_context->misc |= > > cpu_to_le32(NES_QPCONTEXT_MISC_RDMA_WRITE_EN | > > > > NES_QPCONTEXT_MISC_RDMA_READ_EN); > > issue_modify_qp = 1; > > } > > } > There is a comment in our CVS source indicating that was a > workaround for a CM bug in OFED 1.2. We may be able to remove this one. would be nice to fix the underlying bug if it's still present. I would note that cxgb3 has no corresponding code for usermode QPs and appears to work fine. - R. From jackm at dev.mellanox.co.il Mon Jul 21 23:59:44 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 22 Jul 2008 09:59:44 +0300 Subject: [ofa-general] [PATCH 3/3 v2] ib/uverbs: add support for create_qp_expanded in uverbs In-Reply-To: References: Message-ID: <200807220959.44327.jackm@dev.mellanox.co.il> General comment: Please run checkpatch.pl -- there are lots of formatting errors here. See additional comments below. - Jack On Monday 21 July 2008 23:27, ronli at voltaire.com wrote: > -ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, > - const char __user *buf, int in_len, > - int out_len) > +static ssize_t ib_uverbs_create_qp_common(struct ib_uverbs_file *file, > + const char __user *buf, int in_len, > + int out_len, int expanded) > { > - struct ib_uverbs_create_qp cmd; > + struct ib_uverbs_create_qp cmd; unneeded modification > struct ib_uverbs_create_qp_resp resp; > struct ib_udata udata; > struct ib_uqp_object *obj; > @@ -1078,7 +1078,6 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, > goto err_put; > } > > - attr.create_flags = 0; > attr.event_handler = ib_uverbs_qp_event_handler; > attr.qp_context = file; > attr.send_cq = scq; > @@ -1087,7 +1086,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, > attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; > attr.qp_type = cmd.qp_type; > attr.xrc_domain = xrcd; > - attr.create_flags = 0; > + attr.create_flags = expanded ? cmd.reserved : 0; You're still not using a separate cmd structure for ib_uverbs_create_qp_expanded. It is NOT a good idea to use the 8-bit reserved field for ib_uverbs_create_qp_expanded(). You should use a different struct (as I indicated in my previous mail). The common part is more difficult to isolate, since you have different cmd structs, but it is possible. > > @@ -722,6 +726,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, > return ERR_PTR(-ENOMEM); > > err = create_qp_common(dev, pd, init_attr, udata, 0, qp); > + unneeded change > if (err) { > kfree(qp); > return ERR_PTR(err); From jackm at dev.mellanox.co.il Tue Jul 22 00:05:27 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 22 Jul 2008 10:05:27 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 2/3 v2] libmlx4: add support for create_qp_expanded In-Reply-To: References: Message-ID: <200807221005.28017.jackm@dev.mellanox.co.il> On Monday 21 July 2008 23:26, ronli at voltaire.com wrote: > Adds support for the new verb: create_qp_expanded > in libmlx4. > > Changes in v2: > Minimized code duplications by calling mlx4_create_qp_expanded > from mlx4_create_qp. > > +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, > + struct ibv_qp_init_attr *attr, > + uint32_t create_flags) > +{ > + struct mlx4_create_qp cmd; Should be defining and using a struct mlx4_create_qp_expanded here. - Jack From jackm at dev.mellanox.co.il Tue Jul 22 00:09:30 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 22 Jul 2008 10:09:30 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 1/3 v2] libibverbs : add new verbs: create_qp_expanded In-Reply-To: References: Message-ID: <200807221009.30350.jackm@dev.mellanox.co.il> On Monday 21 July 2008 23:25, ronli at voltaire.com wrote: > +struct ibv_create_qp_expanded { > + __u32 command; > + __u16 in_words; > + __u16 out_words; > + __u64 response; > + __u64 user_handle; > + __u32 pd_handle; > + __u32 send_cq_handle; > + __u32 recv_cq_handle; > + __u32 srq_handle; > + __u32 max_send_wr; > + __u32 max_recv_wr; > + __u32 max_send_sge; > + __u32 max_recv_sge; > + __u32 max_inline_data; > + __u8 sq_sig_all; > + __u8 qp_type; > + __u8 is_srq; > + __u8 create_flags; should be: __u8 reserved; __u32 create_flags; __u32 reserved1; The reserved1 field is needed for the proper alignment of the u64 driver_data field in all situations. > + __u64 driver_data[0]; > +}; > + - Jack From sean.hefty at intel.com Tue Jul 22 00:12:19 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 22 Jul 2008 00:12:19 -0700 Subject: [ofa-general] [PATCH] libibcm: get correct ucm device name In-Reply-To: References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> <000201c8eb74$e7b40ed0$64d8180a@amr.corp.intel.com> Message-ID: <000001c8ebca$478615b0$efd8180a@amr.corp.intel.com> Search for the ucm device name that corresponds to a given libibverbs device name. When non-IB devices are present in the system, the ucm device name can differ from the uverbs device name, so we need to lookup the correct name. Signed-off-by: Sean Hefty --- Jeff, can you give this a try and see if it works on your systems? I've tested the patch on my own system, but I only have 1 device in each of my systems. (I'm limited by my hardware here.) If this works, this change, and the other CM change to remove unneeded prints should both go into the next OFED release. diff --git a/src/cm.c b/src/cm.c index c0bcba3..571e28a 100644 --- a/src/cm.c +++ b/src/cm.c @@ -67,6 +67,10 @@ static int abi_ver; static pthread_mutex_t mut = PTHREAD_MUTEX_INITIALIZER; +enum { + IB_UCM_MAX_DEVICES = 32 +}; + #define CM_CREATE_MSG_CMD_RESP(msg, cmd, resp, type, size) \ do { \ struct cm_abi_cmd_hdr *hdr; \ @@ -147,35 +151,64 @@ static int ucm_init(void) return ret; } +static int ucm_get_dev_index(char *dev_name) +{ + char *dev_path; + char ibdev[IBV_SYSFS_NAME_MAX]; + int i, ret; + + for (i = 0; i < IB_UCM_MAX_DEVICES; i++) { + ret = asprintf(&dev_path, "/sys/class/infiniband_cm/ucm%d", i); + if (ret < 0) + return -1; + + ret = ibv_read_sysfs_file(dev_path, "ibdev", ibdev, sizeof ibdev); + if (ret < 0) + continue; + + if (!strcmp(dev_name, ibdev)) { + free(dev_path); + return i; + } + + free(dev_path); + } + return -1; +} + struct ib_cm_device* ib_cm_open_device(struct ibv_context *device_context) { struct ib_cm_device *dev; char *dev_path; + int index, ret; if (ucm_init()) return NULL; + index = ucm_get_dev_index(device_context->device->name); + if (index < 0) + return NULL; + dev = malloc(sizeof *dev); if (!dev) return NULL; dev->device_context = device_context; - if (asprintf(&dev_path, "/dev/infiniband/ucm%s", - device_context->device->dev_name + sizeof("uverbs") - 1) < 0) - goto err2; + ret = asprintf(&dev_path, "/dev/infiniband/ucm%d", index); + if (ret < 0) + goto err1; dev->fd = open(dev_path, O_RDWR); - if (dev->fd < 0) { - goto err; - } + if (dev->fd < 0) + goto err2; free(dev_path); return dev; -err: - free(dev_path); err2: + free(dev_path); +err1: free(dev); return NULL; } From rflores at univaud.com Tue Jul 22 00:25:43 2008 From: rflores at univaud.com (Roderick Flores) Date: Tue, 22 Jul 2008 01:25:43 -0600 Subject: [ofa-general] dhclient will not accept offer In-Reply-To: References: <0E95C6BD-672F-4AFA-B651-8A6506EB3DC9@univaud.com> Message-ID: Thanks for the information. Having said that, we seem to have solved the problem by 1) switching from using /etc/sysconfig/network-scripts/ifcfg-eth0 to / etc/infiniband/ifcfg-ib0. 2) a minor change to the dhcpd.conf file 3) a change to the firewall so that it does not load at boot time. I have since just fixed it. 4) a reboot of both the server and client nodes. I am not exactly sure which piece(s) specifically did the trick. While we do have the system working, dhclient gives the following error-filled output whenever we make a request: /sbin/dhclient-script: configuration for ib0 not found. Continuing with defaults. /etc/sysconfig/network-scripts/network-functions: line 78: ib0: No such file or directory Listening on Socket/ib0 Sending on Socket/ib0 Sending on Socket/fallback DHCPREQUEST on ib0 to 255.255.255.255 port 67 DHCPACK from 172.10.10.2 /sbin/dhclient-script: configuration for ib0 not found. Continuing with defaults. /etc/sysconfig/network-scripts/network-functions: line 78: ib0: No such file or directory bound to 172.10.10.200 -- renewal in 36634 seconds. Linking the ifcfg-ib0 file to the network-scripts directory seems to correct this issue. However the machine will attempt to boot that interface file before the ib_ipoib module is loaded. Although I do not know what the defaults are per se, they seem to work. Informationally, we are using CentOS 5.1 x86_64 with a 2.6.18-53.1.21.el5 kernel. On the client we have the following configuration files... /etc/infiniband/ifcfg-ib0: DEVICE=ib0 ONBOOT=yes BOOTPROTO=dhcp /etc/dhclient.conf: interface "ib0" { send dhcp-client-identifier 20:00:55:00:01:FE: 80:00:00:00:00:00:00:00:11:75:00:00:ff:94:fd; } /etc/modprobe.conf ... alias ib0 ib_ipoib ... /etc/sysconfig/iptables ... -A RH-Firewall-1-INPUT -p udp -m udp --sport 67:68 --dport 67:68 -j ACCEPT ... On the server we have the following configuration files... /etc/dhcpd.conf ddns-update-style interim; ignore client-updates; subnet 172.10.10.0 netmask 255.255.255.0 { default-lease-time 86400; max-lease-time 86400; min-lease-time 3600; option routers 172.10.10.2; option subnet-mask 255.255.248.0; option domain-name "univa.com"; option domain-name-servers 172.10.10.2; range dynamic-bootp 172.10.10.100 172.10.10.200; } /etc/infiniband/ifcfg-eth0 DEVICE=ib0 ONBOOT=yes BOOTPROTO=static IPADDR=172.10.10.2 NETMASK=255.255.255.0 /etc/modprobe.conf ... alias ib0 ib_ipoib ... I have not bothered to configure the firewall on that machine yet. Please feel free to contact me directly if you would like more detailed information. Thanks again for the info... Cheers, Rod On Jul 21, 2008, at 8:16 PM, Dave Olson wrote: > On Mon, 21 Jul 2008, Roderick Flores wrote: > | I created the client identifier from the following information: > | > | 20:<4 byte QP Number><8-byte subnet prefix><8 byte GUID>. > | > | This seems to be in line with the patch that we applied to the > DHCP software > | to make it work with infiniband. > > I've not been able to get dhcp to work with ipoib, but I've not tried > that patch. Has anybody gotten it to work with recent OFED on any IB > HCAs? > > | Lastly, an excerpt from /etc/modules.conf on ip1-test: > | > | alias ib0 ib_ipoib > | > | I tried to use ipath_ether but qlogic's website only has the > RHEL4 release > | (despite having a RHEL5 label). > > It's the same set of rpms for RHEL4 and RHEL5 (identical tarball). > The marketing setup > insists on a per-distro download page, but we don't do independent > builds and testing, just the one RHEL build (which also covers CentOS > and Scientific Linux). > > Dave Olson > dave.olson at qlogic.com From arlin.r.davis at intel.com Tue Jul 22 00:28:08 2008 From: arlin.r.davis at intel.com (Davis, Arlin R) Date: Tue, 22 Jul 2008 00:28:08 -0700 Subject: [ofa-general] [ANNOUNCE] dapl-2.0.11 release - new IB UD extensions Message-ID: New release for uDAPL 2.0 is available on the OFA download page and in my git tree. Added extensions for IB unreliable datagram service Extended EP dat_service_type, with DAT_IB_SERVICE_TYPE_UD Added IB extension call dat_ib_post_send_ud(). Added address handle definition for UD calls. Added IB event definitions to provide remote AH via connect and connect requests See dtestx (-d) source for example usage model md5sum: 7247bade3fc0c075410ee41e386425c8 dapl-2.0.11.tar.gz Summary of changes since last release: v2 - added IB UD extensions, dat_ib_post_send_ud(), new connection events. See http://www.openfabrics.org/downloads/dapl/ more details. Vlad, please pick up this new package for OFED 1.4 alpha. -arlin From vlad at mellanox.co.il Tue Jul 22 00:41:38 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 22 Jul 2008 10:41:38 +0300 Subject: [ofa-general] Re: [ANNOUNCE] dapl-2.0.11 release - new IB UD extensions In-Reply-To: References: Message-ID: <20080722074138.GA25356@mellanox.co.il> On Tue, Jul 22, 2008 at 12:28:08AM -0700, Davis, Arlin R wrote: > > New release for uDAPL 2.0 is available on the OFA download page and in > my git tree. > > Added extensions for IB unreliable datagram service > > Extended EP dat_service_type, with DAT_IB_SERVICE_TYPE_UD > Added IB extension call dat_ib_post_send_ud(). > Added address handle definition for UD calls. > Added IB event definitions to provide remote AH via connect and connect > requests > See dtestx (-d) source for example usage model > > md5sum: 7247bade3fc0c075410ee41e386425c8 dapl-2.0.11.tar.gz > > Summary of changes since last release: > > v2 - added IB UD extensions, dat_ib_post_send_ud(), new connection > events. > > See http://www.openfabrics.org/downloads/dapl/ more details. > > Vlad, please pick up this new package for OFED 1.4 alpha. > > -arlin > Done, Regards, Vladimir From 3rita.fan at trinet.com.tw Mon Jul 21 23:50:14 2008 From: 3rita.fan at trinet.com.tw (griffith hyuk) Date: Tue, 22 Jul 2008 06:50:14 +0000 Subject: [ofa-general] to general Message-ID: <000901c8ebd6$05b21d93$a8f54f88@xhywflvt> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avalwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From vlad at lists.openfabrics.org Tue Jul 22 02:53:46 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 22 Jul 2008 02:53:46 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080722-0200 daily build status Message-ID: <20080722095346.389B1E60331@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Failed: Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.26_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080722-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From acastelfranco at gmail.com Tue Jul 22 03:14:24 2008 From: acastelfranco at gmail.com (Antonino Castelfranco) Date: Tue, 22 Jul 2008 11:14:24 +0100 Subject: [ofa-general] Non-blocking send Message-ID: Hi. I'm trying to understand how to use a non blocking send. Does someone know the simplest way to do it? The scenario is the following: Node A send 10 messages to Node B in a non-blocking way. Sometimes Node A poll the completion queue the verify how many slots in the queue can be freed. Thanks in advance for your help. Antonino Castelfranco Software Engineer From dotanba at gmail.com Tue Jul 22 04:38:47 2008 From: dotanba at gmail.com (Dotan Barak) Date: Tue, 22 Jul 2008 14:38:47 +0300 Subject: [ofa-general] Non-blocking send In-Reply-To: References: Message-ID: <2f3bf9a60807220438j1a4e4b4dv76706b095b1d162d@mail.gmail.com> Hi. On Tue, Jul 22, 2008 at 1:14 PM, Antonino Castelfranco wrote: > Hi. > I'm trying to understand how to use a non blocking send. > Does someone know the simplest way to do it? > The scenario is the following: > > Node A send 10 messages to Node B in a non-blocking way. > Sometimes Node A poll the completion queue the verify how many slots > in the queue can be freed. You should post 10 Send request in the send side (and if needed make sure that there are at least 10 Receive request in the receive side). I thing that the pingpong test can be a good example. Dotan > > Thanks in advance for your help. > > Antonino Castelfranco > Software Engineer > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From ogerlitz at voltaire.com Tue Jul 22 05:23:02 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 22 Jul 2008 15:23:02 +0300 Subject: [ofa-general] Re: [PATCH] ib/ipoib : print error code of ib_sa path lookup call In-Reply-To: References: Message-ID: <4885D126.3090306@voltaire.com> Roland Dreier wrote: > heh, you managed to get a checkpatch error on the only line you changed Oh man, I hope this time I learned the lesson, if not, next time send me to the penalty box... Or. From 7gregz at fathomtech.com Tue Jul 22 04:50:04 2008 From: 7gregz at fathomtech.com (georgy gerry) Date: Tue, 22 Jul 2008 11:50:04 +0000 Subject: [ofa-general] to general Message-ID: <000501c8ec00$070fb7a9$bc0c3c85@dntyneoy> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.avalwosoft[DOT]com (copy this link and then replace "[DOT]" to ".") From rdreier at cisco.com Tue Jul 22 10:31:40 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 10:31:40 -0700 Subject: [ofa-general] Re: [PATCH] mlx4: Add missing FW status return code. In-Reply-To: <200807211806.49276.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Mon, 21 Jul 2008 18:06:48 +0300") References: <200807211806.49276.jackm@dev.mellanox.co.il> Message-ID: thanks, applied. From rdreier at cisco.com Tue Jul 22 11:08:27 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 11:08:27 -0700 Subject: [ofa-general] [PATCH] mlx4_core: Keep free count for MTT buddy allocator Message-ID: MTT entries are allocated with a buddy allocator, which just keeps bitmaps for each level of the buddy table. However, all free space starts out at the highest order, and small allocations start scanning from the lowest order. When the lowest order tables have no free space, this can lead to scanning potentially millions of bits before finding a free entry at a higher order. We can avoid this by just keeping a count of how many free entries each order has, and skipping the bitmap scan when an order is completely empty. This provides a nice performance boost for a negligible increase in memory usage. Signed-off-by: Roland Dreier --- I'll merge this and the corresponding mthca patch for 2.6.27 unless someone sees something wrong with it. drivers/net/mlx4/mlx4.h | 1 + drivers/net/mlx4/mr.c | 26 ++++++++++++++++++-------- 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index a4023c2..7803849 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -118,6 +118,7 @@ struct mlx4_bitmap { struct mlx4_buddy { unsigned long **bits; + unsigned int *num_free; int max_order; spinlock_t lock; }; diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 03a9abc..b3ea93b 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -79,23 +79,26 @@ static u32 mlx4_buddy_alloc(struct mlx4_buddy *buddy, int order) spin_lock(&buddy->lock); - for (o = order; o <= buddy->max_order; ++o) { - m = 1 << (buddy->max_order - o); - seg = find_first_bit(buddy->bits[o], m); - if (seg < m) - goto found; - } + for (o = order; o <= buddy->max_order; ++o) + if (buddy->num_free[o]) { + m = 1 << (buddy->max_order - o); + seg = find_first_bit(buddy->bits[o], m); + if (seg < m) + goto found; + } spin_unlock(&buddy->lock); return -1; found: clear_bit(seg, buddy->bits[o]); + --buddy->num_free[o]; while (o > order) { --o; seg <<= 1; set_bit(seg ^ 1, buddy->bits[o]); + ++buddy->num_free[o]; } spin_unlock(&buddy->lock); @@ -113,11 +116,13 @@ static void mlx4_buddy_free(struct mlx4_buddy *buddy, u32 seg, int order) while (test_bit(seg ^ 1, buddy->bits[order])) { clear_bit(seg ^ 1, buddy->bits[order]); + --buddy->num_free[order]; seg >>= 1; ++order; } set_bit(seg, buddy->bits[order]); + ++buddy->num_free[order]; spin_unlock(&buddy->lock); } @@ -131,7 +136,9 @@ static int mlx4_buddy_init(struct mlx4_buddy *buddy, int max_order) buddy->bits = kzalloc((buddy->max_order + 1) * sizeof (long *), GFP_KERNEL); - if (!buddy->bits) + buddy->num_free = kzalloc((buddy->max_order + 1) * sizeof (int *), + GFP_KERNEL); + if (!buddy->bits || !buddy->num_free) goto err_out; for (i = 0; i <= buddy->max_order; ++i) { @@ -143,6 +150,7 @@ static int mlx4_buddy_init(struct mlx4_buddy *buddy, int max_order) } set_bit(0, buddy->bits[buddy->max_order]); + buddy->num_free[buddy->max_order] = 1; return 0; @@ -150,9 +158,10 @@ err_out_free: for (i = 0; i <= buddy->max_order; ++i) kfree(buddy->bits[i]); +err_out: kfree(buddy->bits); + kfree(buddy->num_free); -err_out: return -ENOMEM; } @@ -164,6 +173,7 @@ static void mlx4_buddy_cleanup(struct mlx4_buddy *buddy) kfree(buddy->bits[i]); kfree(buddy->bits); + kfree(buddy->num_free); } static u32 mlx4_alloc_mtt_range(struct mlx4_dev *dev, int order) -- 1.5.6.2 From rdreier at cisco.com Tue Jul 22 11:09:20 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 11:09:20 -0700 Subject: [ofa-general] [PATCH] IB/mthca: Keep free count for MTT buddy allocator Message-ID: MTT entries are allocated with a buddy allocator, which just keeps bitmaps for each level of the buddy table. However, all free space starts out at the highest order, and small allocations start scanning from the lowest order. When the lowest order tables have no free space, this can lead to scanning potentially millions of bits before finding a free entry at a higher order. We can avoid this by just keeping a count of how many free entries each order has, and skipping the bitmap scan when an order is completely empty. This provides a nice performance boost for a negligible increase in memory usage. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mthca/mthca_dev.h | 1 + drivers/infiniband/hw/mthca/mthca_mr.c | 26 ++++++++++++++++++-------- 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_dev.h b/drivers/infiniband/hw/mthca/mthca_dev.h index ee4d073..2525901 100644 --- a/drivers/infiniband/hw/mthca/mthca_dev.h +++ b/drivers/infiniband/hw/mthca/mthca_dev.h @@ -202,6 +202,7 @@ struct mthca_pd_table { struct mthca_buddy { unsigned long **bits; + int *num_free; int max_order; spinlock_t lock; }; diff --git a/drivers/infiniband/hw/mthca/mthca_mr.c b/drivers/infiniband/hw/mthca/mthca_mr.c index 8489b1e..882e6b7 100644 --- a/drivers/infiniband/hw/mthca/mthca_mr.c +++ b/drivers/infiniband/hw/mthca/mthca_mr.c @@ -89,23 +89,26 @@ static u32 mthca_buddy_alloc(struct mthca_buddy *buddy, int order) spin_lock(&buddy->lock); - for (o = order; o <= buddy->max_order; ++o) { - m = 1 << (buddy->max_order - o); - seg = find_first_bit(buddy->bits[o], m); - if (seg < m) - goto found; - } + for (o = order; o <= buddy->max_order; ++o) + if (buddy->num_free[o]) { + m = 1 << (buddy->max_order - o); + seg = find_first_bit(buddy->bits[o], m); + if (seg < m) + goto found; + } spin_unlock(&buddy->lock); return -1; found: clear_bit(seg, buddy->bits[o]); + --buddy->num_free[o]; while (o > order) { --o; seg <<= 1; set_bit(seg ^ 1, buddy->bits[o]); + ++buddy->num_free[o]; } spin_unlock(&buddy->lock); @@ -123,11 +126,13 @@ static void mthca_buddy_free(struct mthca_buddy *buddy, u32 seg, int order) while (test_bit(seg ^ 1, buddy->bits[order])) { clear_bit(seg ^ 1, buddy->bits[order]); + --buddy->num_free[order]; seg >>= 1; ++order; } set_bit(seg, buddy->bits[order]); + ++buddy->num_free[order]; spin_unlock(&buddy->lock); } @@ -141,7 +146,9 @@ static int mthca_buddy_init(struct mthca_buddy *buddy, int max_order) buddy->bits = kzalloc((buddy->max_order + 1) * sizeof (long *), GFP_KERNEL); - if (!buddy->bits) + buddy->num_free = kzalloc((buddy->max_order + 1) * sizeof (int *), + GFP_KERNEL); + if (!buddy->bits || !buddy->num_free) goto err_out; for (i = 0; i <= buddy->max_order; ++i) { @@ -154,6 +161,7 @@ static int mthca_buddy_init(struct mthca_buddy *buddy, int max_order) } set_bit(0, buddy->bits[buddy->max_order]); + buddy->num_free[buddy->max_order] = 1; return 0; @@ -161,9 +169,10 @@ err_out_free: for (i = 0; i <= buddy->max_order; ++i) kfree(buddy->bits[i]); +err_out: kfree(buddy->bits); + kfree(buddy->num_free); -err_out: return -ENOMEM; } @@ -175,6 +184,7 @@ static void mthca_buddy_cleanup(struct mthca_buddy *buddy) kfree(buddy->bits[i]); kfree(buddy->bits); + kfree(buddy->num_free); } static u32 mthca_alloc_mtt_range(struct mthca_dev *dev, int order, -- 1.5.6.2 From Nathan.Dauchy at noaa.gov Tue Jul 22 11:31:39 2008 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Tue, 22 Jul 2008 12:31:39 -0600 Subject: [ofa-general] OpenSM "Dead end on path to LID" In-Reply-To: <48811106.9030906@dev.mellanox.co.il> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> <4880C61A.6010000@noaa.gov> <48811106.9030906@dev.mellanox.co.il> Message-ID: <4886278B.2080502@noaa.gov> Yevgeny, and anyone else with ideas on this problem, Yevgeny Kliteynik wrote: > Nathan Dauchy wrote: >> Yevgeny Kliteynik wrote: >>> I understand you already increased transaction time. >>> Please try limiting SMPs on the wire - in opensm.conf >>> file, set max_wire_smps to 1 (you probably have 4). >>> You can also run opensm with '-maxsmps 1' command line >>> argument. >> >> Interesting! >> >> I believe MAXSMPS was originally set to 0 (unlimited), based on >> duplicating the config file from an older SM setup. I reduced it to 32 >> when we saw some IB errors on standalone System B. I'm afraid I don't >> have documentation on what those problems were, but I don't recall >> seeing the exact same symptoms. I think it was MAD timeout error >> messages that prompted me to change the MAXSMPS value. > > You probably have the timeout errors in the osm log now as well. > These errors are followed by the "unknown remote side" messages, > which means that osm didn't get response to some MADs (timeout). > >> We will be able to test any fixes during a scheduled system downtime on >> 7/24. At that point, do you recommend trying MAXSMPS=4? > > In ofed 1.3.1, the default conf file value for maxsmps should be 4. > If you're using 0, then I'm pretty sure that this is what causing > the problems. Try 4 - it should be fine. We went ahead and tried both MAXSMPS=4 and MAXSMPS=1. The symptoms did not improve with all the nodes booted. :( For the record, here is exactly how opensm is running now: # ps uaxw | grep open root 23112 1.0 0.1 288432 16732 ? Sl 18:10 0:07 /opt/ofed/1.3.1/sbin/opensm -maxsmps 1 -t 600 -f /var/run/osm/osm.log -R updn -g 0 --honor_guid2lid >> If that doesn't work, what else should we be prepared to try, or what >> other debugging information would be helpful to gather? >> >> For the record, other steps we are considering: >> * Latest OpenSM code >> * Upgrading firmware on all IB switches >> * Changing topology to remove the 12X links (ugh!) > > Hope that there won't be a need for other steps. To try to take the 12X links out of the equation, we replaced the (3) "Subtree C" spine switches with ones burned with the default 24*4X .INI file, and removed 2 cables from each 3-cable bundle. The "Subtree C" Clos/Edge switches and Root/Aggregation switches were still configured as 12X links, but according to "ibdiagnet", they negotiated down to 4X. (I can furnish a nicer diagram of our IB tree off-list if anyone would like to take a look at it.) Any other things we can try? Thanks again, Nathan From rdreier at cisco.com Tue Jul 22 11:35:40 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 11:35:40 -0700 Subject: [ofa-general] [PATCH/RFC] mlx4_core: Improve error message when not enough UAR pages are available Message-ID: If an mlx4 device with default FW (which gives a UAR BAR size of 8 MB) is used in a system with 64 KB pages, then there are only 8192/64==128 UAR pages available. However, the first 128 UAR pages are reserved for use with event queue doorbells, so no UAR pages are available to do anything else with, which means that the driver cannot work. The current driver fails with a fairly cryptic "Failed to allocate driver access region, aborting" message in this situation. Fix the driver to detect the problem earlier and print out a clearer description of the problem and a suggestion of how to fix it (use a new firmware image). Signed-off-by: Roland Dreier --- I helped debug this situation recently, and I think this improved error reporting may help in the future. Any objections to merging this? drivers/net/mlx4/pd.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/drivers/net/mlx4/pd.c b/drivers/net/mlx4/pd.c index 3a93c5f..aa61689 100644 --- a/drivers/net/mlx4/pd.c +++ b/drivers/net/mlx4/pd.c @@ -91,6 +91,13 @@ EXPORT_SYMBOL_GPL(mlx4_uar_free); int mlx4_init_uar_table(struct mlx4_dev *dev) { + if (dev->caps.num_uars <= 128) { + mlx4_err(dev, "Only %d UAR pages (need more than 128)\n", + dev->caps.num_uars); + mlx4_err(dev, "Increase firmware log2_uar_bar_megabytes?\n"); + return -ENODEV; + } + return mlx4_bitmap_init(&mlx4_priv(dev)->uar_table.bitmap, dev->caps.num_uars, dev->caps.num_uars - 1, max(128, dev->caps.reserved_uars)); -- 1.5.6.2 From jsquyres at cisco.com Tue Jul 22 11:47:05 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Tue, 22 Jul 2008 14:47:05 -0400 Subject: [ofa-general] Re: [PATCH] libibcm: get correct ucm device name In-Reply-To: <000001c8ebca$478615b0$efd8180a@amr.corp.intel.com> References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> <000201c8eb74$e7b40ed0$64d8180a@amr.corp.intel.com> <000001c8ebca$478615b0$efd8180a@amr.corp.intel.com> Message-ID: Sean -- I have since rebooted my one node that got the ordering wrong upon startup, and now despite many reboots on all my nodes that have an NE020 and ConnectX HCA (3), I cannot get the ordering to be "wrong" again -- it always puts ConnectX first, so the original problem doesn't surface. :-\ But FWIW, this patch seems to work fine for me in the scenarios where the IB device is first. Thanks! On Jul 22, 2008, at 3:12 AM, Sean Hefty wrote: > Search for the ucm device name that corresponds to a given libibverbs > device name. When non-IB devices are present in the system, the ucm > device name can differ from the uverbs device name, so we need to > lookup the correct name. > > Signed-off-by: Sean Hefty > --- > Jeff, can you give this a try and see if it works on your systems? > I've > tested the patch on my own system, but I only have 1 device in each > of my > systems. (I'm limited by my hardware here.) If this works, this > change, > and the other CM change to remove unneeded prints should both go into > the next OFED release. > > diff --git a/src/cm.c b/src/cm.c > index c0bcba3..571e28a 100644 > --- a/src/cm.c > +++ b/src/cm.c > @@ -67,6 +67,10 @@ > static int abi_ver; > static pthread_mutex_t mut = PTHREAD_MUTEX_INITIALIZER; > > +enum { > + IB_UCM_MAX_DEVICES = 32 > +}; > + > #define CM_CREATE_MSG_CMD_RESP(msg, cmd, resp, type, size) \ > do { \ > struct cm_abi_cmd_hdr *hdr; \ > @@ -147,35 +151,64 @@ static int ucm_init(void) > return ret; > } > > +static int ucm_get_dev_index(char *dev_name) > +{ > + char *dev_path; > + char ibdev[IBV_SYSFS_NAME_MAX]; > + int i, ret; > + > + for (i = 0; i < IB_UCM_MAX_DEVICES; i++) { > + ret = asprintf(&dev_path, "/sys/class/infiniband_cm/ucm%d", i); > + if (ret < 0) > + return -1; > + > + ret = ibv_read_sysfs_file(dev_path, "ibdev", ibdev, sizeof ibdev); > + if (ret < 0) > + continue; > + > + if (!strcmp(dev_name, ibdev)) { > + free(dev_path); > + return i; > + } > + > + free(dev_path); > + } > + return -1; > +} > + > struct ib_cm_device* ib_cm_open_device(struct ibv_context > *device_context) > { > struct ib_cm_device *dev; > char *dev_path; > + int index, ret; > > if (ucm_init()) > return NULL; > > + index = ucm_get_dev_index(device_context->device->name); > + if (index < 0) > + return NULL; > + > dev = malloc(sizeof *dev); > if (!dev) > return NULL; > > dev->device_context = device_context; > > - if (asprintf(&dev_path, "/dev/infiniband/ucm%s", > - device_context->device->dev_name + sizeof("uverbs") - 1) < 0) > - goto err2; > + ret = asprintf(&dev_path, "/dev/infiniband/ucm%d", index); > + if (ret < 0) > + goto err1; > > dev->fd = open(dev_path, O_RDWR); > - if (dev->fd < 0) { > - goto err; > - } > + if (dev->fd < 0) > + goto err2; > > free(dev_path); > return dev; > > -err: > - free(dev_path); > err2: > + free(dev_path); > +err1: > free(dev); > return NULL; > } > > > -- Jeff Squyres Cisco Systems From sean.hefty at intel.com Tue Jul 22 11:55:45 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 22 Jul 2008 11:55:45 -0700 Subject: [ofa-general] RE: [PATCH] libibcm: get correct ucm device name In-Reply-To: References: <2D02DCB0-237E-46CD-A113-565048638A0E@cisco.com> <000201c8eb74$e7b40ed0$64d8180a@amr.corp.intel.com> <000001c8ebca$478615b0$efd8180a@amr.corp.intel.com> Message-ID: <000001c8ec2c$8c83c060$17248686@amr.corp.intel.com> >I have since rebooted my one node that got the ordering wrong upon >startup, and now despite many reboots on all my nodes that have an >NE020 and ConnectX HCA (3), I cannot get the ordering to be "wrong" >again -- it always puts ConnectX first, so the original problem >doesn't surface. :-\ > >But FWIW, this patch seems to work fine for me in the scenarios where >the IB device is first. Thanks for testing it. At least it doesn't look like we're worse off. I will go ahead and commit the patch to my tree and add it to the next release. - Sean From rgekw at mail.navehumor.com Tue Jul 22 14:09:28 2008 From: rgekw at mail.navehumor.com (patrice) Date: Tue, 22 Jul 2008 13:09:28 -0800 (EDT) Subject: [ofa-general] is it you? patrice here Message-ID: <75020490.2742401362980.JavaMail.vmail@service3.colo.trueswitch.com> Hi, i am here sitting in the internet caffe. Found your email and decided to write. I am 25 y.o.girl. I have a picture if you want. No need to reply here as this is not may email. Write me at apatrice37 at gblcentral.info From mjtysml at crewstart.com Tue Jul 22 14:14:05 2008 From: mjtysml at crewstart.com (janet) Date: Tue, 22 Jul 2008 13:14:05 -0800 Subject: [ofa-general] hi from janet Message-ID: Do not ignore me please, I found your email somewhere and now decided to write you. Let me know if you do not mind. If you want I can send you some pictures of me. I am a nice pretty girl. Don't reply to this email. Email me direclty at djanet at onlineldp.info From preselected at campinghecho.com Tue Jul 22 15:49:59 2008 From: preselected at campinghecho.com (Molly Nelson) Date: Tue, 22 Jul 2008 13:49:59 -0900 Subject: [ofa-general] Acrobat 9 Pro Extended Message-ID: <000501c8ec3b$8e9df000$0100007f@awmhts> ^The following titles and more are now available for pcs and macs- CyberLink PowerDVD ULTRA EDITION v8 Office 2008 Special Media Edition Traktor DJ Studio Acrobat 9 Pro Extended CorelDRAW Graphics Suite X4 CorelDRAW Graphics Suite X4 ^ softwaredeals4less . com - System Requirements - For PC: - Intel Pentium 4 (1.4GHz processor), Intel Centrino, Intel Xeon CorelDRAW Graphics Suite X4 or Intel Core Duo (or CorelDRAW Graphics Suite X4) processor; SSE2-enabled processor required for AMD systems & Microsoft Windows XP with Service Pack 2 or Microsoft Windows Vista Home Premium, Business, Ultimate, or Enterprise (certified for 32-bit editions) - 512MB of RAM or more - 1GB of available hard-disk space (additional free space CorelDRAW Graphics Suite X4) - Microsoft compatible sound card (multichannel ASIO-compatible sound card recommended) - 1,024x768 monitor resolution with 32-bit color adapter recommended - CD-R or DVD-ROM drive ^ For MAC: ^ PowerPC G4 or G5 or multicore Intel processor ^ Mac OS X or similar ^ 512MB of RAM or more ^ 1GB of available hard-disk space ^ Core Audio compatible sound card ^ 1,024x768 monitor resolution with 32-bit color adapter ^ DVD-ROM drive- DVD+-R burner required for DVD creation International authorities have finally captured former Bosnian Serb leader Radovan Karadzic, one of the most wanted men in the world. They'd been chasing him for over a decade on genocide charges. From rdreier at cisco.com Tue Jul 22 14:05:01 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 14:05:01 -0700 Subject: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: <48849F59.7060502@Voltaire.COM> (Yossi Etigin's message of "Mon, 21 Jul 2008 17:38:17 +0300") References: <48849F59.7060502@Voltaire.COM> Message-ID: > The problem: > A multicast sender joins the MGID as full member, but does not leave > (as long as the interface is up). This causes an MGID leakage in the > SM. I think ipoib has used send-only joins for a while now. But the problem of never cleaning up these memberships stands. Thanks for looking at this -- it would be nice to deal with this. > An issue we found with this solution, is that it takes a while until the > neighbour is cleaned, and it requires that the host will send some traffic > (I supposed that unless we put more entries in the routing table, out neighbour > will not be garbage collected). Would it work to have a more naive approach, and just keep track of the last time each send-only membership was used, and set up a timer that runs, say every minute, and garbage collect send-only memberships that are more than 2 minutes old, or something like that? - R. From rdreier at cisco.com Tue Jul 22 14:12:25 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 14:12:25 -0700 Subject: [ofa-general] Re: [PATCH v2 3/4] rdma/cma: implement RDMA_CM_EVENT_ADDR_CHANGE notification In-Reply-To: (Or Gerlitz's message of "Thu, 19 Jun 2008 17:07:27 +0300 (IDT)") References: Message-ID: thanks, I applied this. I noticed that the RDMA_CM_EVENT_ADDR_CHANGE conflicted with the RDMA_CM_EVENT_TIMEWAIT_EXIT added by the earlier patch. Then I looked at the librdmacm git tree and saw that ADDR_CHANGE is supposed to come first, so I reordered the patches to add ADDR_CHANGE and then add TIMEWAIT_EXIT after it. - R. From rdreier at cisco.com Tue Jul 22 14:14:32 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 14:14:32 -0700 Subject: [ofa-general] [PATCH v2 3/4] ib/iser: add support for the ADDR_CHANGE rdma-cm event In-Reply-To: (Or Gerlitz's message of "Thu, 19 Jun 2008 17:07:56 +0300 (IDT)") References: Message-ID: thanks, applied From mashirle at us.ibm.com Tue Jul 22 15:06:50 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Tue, 22 Jul 2008 15:06:50 -0700 Subject: [ofa-general] [PATCH] libibverbs: Replace eieio with sync for PPC wmb() Message-ID: <1216764410.31058.11.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, We have found that the wmb() for PPC was incorrect defined as eieio instruction in libibverbs. Instruction eieio applies either in a pure I/O memory or a pure system memory. In the situation where the device drivers use the d_map kernel services to share a portion of system memory with an I/O adapter, we need to use sync() instead. See below link for reference. http://www.ibm.com/developerworks/eserver/articles/powerpc.html Signed-off-by: Shirley Ma ------- include/infiniband/arch.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/include/infiniband/arch.h b/include/infiniband/arch.h index 6931bfc..d3e356f 100644 --- a/include/infiniband/arch.h +++ b/include/infiniband/arch.h @@ -98,7 +98,7 @@ static inline uint64_t ntohll(uint64_t x) { return x; } #define mb() asm volatile("sync" ::: "memory") #define rmb() mb() -#define wmb() asm volatile("eieio" ::: "memory") +#define wmb() mb() #define wc_wmb() wmb() #elif defined(__sparc_v9__) From or.gerlitz at gmail.com Tue Jul 22 14:17:47 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Wed, 23 Jul 2008 00:17:47 +0300 Subject: ***SPAM*** Re: [ofa-general] Re: [PATCH v2 3/4] rdma/cma: implement RDMA_CM_EVENT_ADDR_CHANGE notification In-Reply-To: References: Message-ID: <15ddcffd0807221417k5dbaf6f4na018b964a1862e9b@mail.gmail.com> On Wed, Jul 23, 2008 at 12:12 AM, Roland Dreier wrote: > I looked at the librdmacm git tree and saw that ADDR_CHANGE > is supposed to come first, so I reordered the patches to add ADDR_CHANGE > and then add TIMEWAIT_EXIT after it. yes, I've dropped you a note on this couple of days ago http://lists.openfabrics.org/pipermail/general/2008-July/052951.html but you might missed it, thanks for fixing that. Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mashirle at us.ibm.com Tue Jul 22 17:21:22 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Tue, 22 Jul 2008 17:21:22 -0700 Subject: [ofa-general] Quetions about IPOIB handle last WQE event Message-ID: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, Eli, We have seen heavy QP resource leakages in ehca for both nonSRQ and SRQ for IPoIB-CM mode. I have several patches to fix this issue. Before I submit these patches for review. I would like to discuss current IPoIB resource release based on last WQE first. In IB spec Section 10.3.1 -------------------------- Note, for QPs that are associated with an SRQ, the Consumer should take the QP through the Error State before invoking a Destroy QP or a Modify QP to the Reset State. The Consumer may invoke the Destroy QP without first performing a Modify QP to the Error State and waiting for the Affiliated Asynchronous Last WQE Reached Event. However, if the Consumer does not wait for the Affiliated Asynchronous Last WQE Reached Event, then WQE and Data Segment leakage may occur. Therefore, it is programming practice to tear down a QP that is associated with an SRQ by using the following process: • Put the QP in the Error State; • wait for the Affiliated Asynchronous Last WQE Reached Event; • either: • drain the CQ by invoking the Poll CQ verb and either wait for CQ to be empty or the number of Poll CQ operations has exceeded CQ capacity size; or • post another WR that completes on the same CQ and wait for this WR to return as a WC; • and then invoke a Destroy QP or Reset QP. Section: 11-5.2.5: ------------------ If the HCA supports SRQ, for RC and UD service, the CI shall generate a Last WQE Reached Affiliated Asynchronous Event on a QP that is in the Error State and is associated with an SRQ when either: • a CQE is generated for the last WQE, or • the QP gets in the Error State and there are no more WQEs on the RQ. The IPoIB-CM implementation takes the approach by posting another WR that completes on the same CQ and wait for this WR to return as a WC. IPoIB first puts the QP in error status, then waits for last WQE event in async event handler by posting a drain WR, the QP resources will be released in when last CEQs being generated. However it works for ConnectionX but not for ehca. In ehca implemention it follows Section 11-5.2.5, when the QP gets in the Error state, and there are no more WQEs on the RQ. So these QP resources are never being released thus causes QP resources leakage, no QPs can't be released at all. So when the maxium QPs are reached (default nonSRQ is 128, SRQ is 4K), no more new connections can be built. Nodes can't even be reachable. We can see this problem even in an idle cluster. In an idle cluster, if we have occasional ping, like ping every 10mins. The arp entry life time is around 6 mins by default if no more traffics, (route cache timeout 300s + random(1/2 * reachable_time(15s), 3/2* reachable_time) + gc clean(60s)) Then arp entries would be expired, a neigh detroy will be called from neigh garbage collection, then ipoib neigh cleanup gets called. It will destroy tx QPs and destroy cm_id will send DREQ to remote connection. The remote side after receives DREQ, it puts the QP to error status. Then the last WQE reached event will be generated. node-1 node-2 --------------------------------------- ping -c 1 node-2 TX QP0 create arp entry RX QP0 create wait for 10 mins arp entry is released ipoib_neigh_cleanup destroy TX QP destroy cm_id send DREQ received DREQ put QP0 in error status wait for async event LAST WQE reached event for RX QP0 post last WR for QP0 poll_cq below only applies to Mellanox, ehca won't see last WQ in SRQ ---------------- see last WR for QP0 put QP0 in reap_list for clean up queue reap work reap work: clean QP0 ----------------- ehca still keeps QP0 Repeat above steps in a large cluster, number of RX QPs will eventually run out for SRQ. Since nonSRQ doesn't handle async event, it never releases QPs, 128 connections will run out soon even in a two nodes cluster by repeating above steps. ( This is another bug, I will submit a fix). The above approach has a couple issues: 1. It works only for mthca/mlx4 not for ehca 2. If node-1 fails to send DREQ for any reason to remote, like node-1 shutdown, then RX QP in node-2 will be put in the error list after around 21 mins (IPOIB_CM_RX_TIMEOUT + IPOIB_CM_RX_DELAY 5*256*HZ) #define IPOIB_CM_RX_TIMEOUT (2 * 256 * HZ) #define IPOIB_CM_RX_DELAY (3 * 256 * HZ)) The timer seems too long for release stale QP resources, we could hit QP run out in a large cluster even for mthca/mlx4. My questions here are: 1. Whether it's a MUST to put QP in error status before posting last WR? if it's a MUST, why? 2. Last WQE event is only generated once for each QP even IPoIB sets QP into error status and the CI surfaced a Local Work Queue Catastrophic Error on the same QP at the same time, is that right? I will post my patchsets for review based on this discussion outcome. Thanks Shirley From weiny2 at llnl.gov Tue Jul 22 17:28:09 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 22 Jul 2008 17:28:09 -0700 Subject: [ofa-general] [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM Message-ID: <20080722172809.5d0831f3.weiny2@llnl.gov> >From be896a92f5e78fc315b4dbfcb77e7d8f07612769 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 22 Jul 2008 16:31:36 -0700 Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 43 +++++++++++++++++++++++++++++++++++++++++ opensm/opensm/osm_inform.c | 3 +- opensm/opensm/osm_trap_rcv.c | 10 +++++++- 3 files changed, 53 insertions(+), 3 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 09ec257..204253c 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7219,6 +7219,49 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) +static inline char * OSM_API +ib_trap_str(uint16_t trap_num) +{ + switch(trap_num) + { + case 64: + return ("GID in service"); + case 65: + return ("GID out of service"); + case 66: + return ("New mcast group created"); + case 67: + return ("Mcast group deleted"); + case 68: + return ("UnPath, Path no longer valid"); + case 69: + return ("RePath, Path recomputed"); + case 128: + return ("Link state change"); + case 129: + return ("Local Link integrity threshold reached"); + case 130: + return ("Excessive Buffer Overrun Threshold reached"); + case 131: + return ("Flow Control Update watchdog timer expired"); + case 144: + return ("CapabilityMask, NodeDescriptor, Link [Width|Speed] Enabled changed"); + case 145: + return ("System Image GUID changed"); + case 256: + return ("Bad M_Key"); + case 257: + return ("Bad P_Key"); + case 258: + return ("Bad Q_Key"); + case 259: + return ("Bad P_Key (switch external port)"); + default: + break; + } + return ("Unknown"); +} + /****f* IBA Base: Types/ib_notice_is_generic * NAME * ib_notice_is_generic diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index e6146f7..77dad32 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, /* an official Event information log */ if (ib_notice_is_generic(p_ntc)) { OSM_LOG(p_log, OSM_LOG_INFO, - "Reporting Generic Notice type:%u num:%u" + "Reporting Generic Notice type:%u num:%u (%s)" " from LID:%u GID:0x%016" PRIx64 ",0x%016" PRIx64 "\n", ib_notice_get_type(p_ntc), cl_ntoh16(p_ntc->g_or_v.generic.trap_num), + ib_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), cl_ntoh16(p_ntc->issuer_lid), cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index ae7ab05..bcb4260 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, || (p_ntci->g_or_v.generic.trap_num == CL_HTON16(131))) OSM_LOG(sm->p_log, OSM_LOG_ERROR, - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " + "Received Generic Notice type:0x%02X " + "num:%u (%s) Producer:%u (%s) " "from LID:%u Port %d TID:0x%016" PRIx64 "\n", ib_notice_get_type(p_ntci), cl_ntoh16(p_ntci->g_or_v.generic. trap_num), + ib_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. + trap_num)), cl_ntoh32(ib_notice_get_prod_type (p_ntci)), ib_get_producer_type_str @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, ); else OSM_LOG(sm->p_log, OSM_LOG_ERROR, - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " + "Received Generic Notice type:0x%02X " + "num:%u (%s) Producer:%u (%s) " "from LID:%u TID:0x%016" PRIx64 "\n", ib_notice_get_type(p_ntci), cl_ntoh16(p_ntci->g_or_v.generic. trap_num), + ib_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. + trap_num)), cl_ntoh32(ib_notice_get_prod_type (p_ntci)), ib_get_producer_type_str -- 1.5.4.5 From hal.rosenstock at gmail.com Tue Jul 22 18:06:40 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 22 Jul 2008 21:06:40 -0400 Subject: ***SPAM*** Re: [ofa-general] [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080722172809.5d0831f3.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> Message-ID: One nit below... On Tue, Jul 22, 2008 at 8:28 PM, Ira Weiny wrote: > >From be896a92f5e78fc315b4dbfcb77e7d8f07612769 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 22 Jul 2008 16:31:36 -0700 > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM > > > Signed-off-by: Ira K. Weiny > --- > opensm/include/iba/ib_types.h | 43 +++++++++++++++++++++++++++++++++++++++++ > opensm/opensm/osm_inform.c | 3 +- > opensm/opensm/osm_trap_rcv.c | 10 +++++++- > 3 files changed, 53 insertions(+), 3 deletions(-) > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > index 09ec257..204253c 100644 > --- a/opensm/include/iba/ib_types.h > +++ b/opensm/include/iba/ib_types.h > @@ -7219,6 +7219,49 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) > #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) > > +static inline char * OSM_API > +ib_trap_str(uint16_t trap_num) > +{ > + switch(trap_num) > + { > + case 64: > + return ("GID in service"); > + case 65: > + return ("GID out of service"); > + case 66: > + return ("New mcast group created"); > + case 67: > + return ("Mcast group deleted"); > + case 68: > + return ("UnPath, Path no longer valid"); > + case 69: > + return ("RePath, Path recomputed"); > + case 128: > + return ("Link state change"); > + case 129: > + return ("Local Link integrity threshold reached"); > + case 130: > + return ("Excessive Buffer Overrun Threshold reached"); > + case 131: > + return ("Flow Control Update watchdog timer expired"); > + case 144: > + return ("CapabilityMask, NodeDescriptor, Link [Width|Speed] ^^^^^^^^^^^ Description Enabled changed"); > + case 145: > + return ("System Image GUID changed"); > + case 256: > + return ("Bad M_Key"); > + case 257: > + return ("Bad P_Key"); > + case 258: > + return ("Bad Q_Key"); > + case 259: > + return ("Bad P_Key (switch external port)"); > + default: > + break; > + } > + return ("Unknown"); > +} > + > /****f* IBA Base: Types/ib_notice_is_generic > * NAME > * ib_notice_is_generic > diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c > index e6146f7..77dad32 100644 > --- a/opensm/opensm/osm_inform.c > +++ b/opensm/opensm/osm_inform.c > @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, > /* an official Event information log */ > if (ib_notice_is_generic(p_ntc)) { > OSM_LOG(p_log, OSM_LOG_INFO, > - "Reporting Generic Notice type:%u num:%u" > + "Reporting Generic Notice type:%u num:%u (%s)" > " from LID:%u GID:0x%016" PRIx64 > ",0x%016" PRIx64 "\n", > ib_notice_get_type(p_ntc), > cl_ntoh16(p_ntc->g_or_v.generic.trap_num), > + ib_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), > cl_ntoh16(p_ntc->issuer_lid), > cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), > cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) > diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c > index ae7ab05..bcb4260 100644 > --- a/opensm/opensm/osm_trap_rcv.c > +++ b/opensm/opensm/osm_trap_rcv.c > @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > || (p_ntci->g_or_v.generic.trap_num == > CL_HTON16(131))) > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " > + "Received Generic Notice type:0x%02X " > + "num:%u (%s) Producer:%u (%s) " > "from LID:%u Port %d TID:0x%016" > PRIx64 "\n", ib_notice_get_type(p_ntci), > cl_ntoh16(p_ntci->g_or_v.generic. > trap_num), > + ib_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. > + trap_num)), > cl_ntoh32(ib_notice_get_prod_type > (p_ntci)), > ib_get_producer_type_str > @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > ); > else > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " > + "Received Generic Notice type:0x%02X " > + "num:%u (%s) Producer:%u (%s) " > "from LID:%u TID:0x%016" PRIx64 > "\n", ib_notice_get_type(p_ntci), > cl_ntoh16(p_ntci->g_or_v.generic. > trap_num), > + ib_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. > + trap_num)), > cl_ntoh32(ib_notice_get_prod_type > (p_ntci)), > ib_get_producer_type_str > -- > 1.5.4.5 > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From flatif at neteffect.com Tue Jul 22 18:22:34 2008 From: flatif at neteffect.com (Faisal Latif) Date: Tue, 22 Jul 2008 20:22:34 -0500 Subject: [ofa-general] ***SPAM*** [PATCH] RDMA/nes: CM connection setup/teardown rework Message-ID: <200807230122.m6N1MYeY003840@velma.neteffect.com> Major rework of CM connection setup/teardown. We had a number of issues with MPI applications not starting/terminating properly over time. With these changes we were able to run longer on larger clusters. * Removed memory allocation from nes_connect() and nes_cm_connect(). * Fixed mini_cm_dec_refcnt_listen() when destroying listener. * Removed unnecessary code from schedule_nes_timer() and nes_cm_timer_tick(). * Functionalized mini_cm_recv_pkt() and process_packet(). * Cleaned up cm_node->ref_count usage. * Reuse skbs if available. Signed-off-by: Faisal Latif --- We have a few more CM changes planned for 2.6.27: * Modify nes_cm_timer_tick() to use separate timer pending cm_node list. * Modify async event handling in nes_process_iwarp_aeqe(). * Create pending listener list for netdev/CM node with invalid IPs. diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index 6aa531d..f658ab7 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -74,36 +74,59 @@ atomic_t cm_nodes_destroyed; atomic_t cm_accel_dropped_pkts; atomic_t cm_resets_recvd; -static inline int mini_cm_accelerated(struct nes_cm_core *, struct nes_cm_node *); +static inline int mini_cm_accelerated(struct nes_cm_core *, + struct nes_cm_node *); static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *, - struct nes_vnic *, struct nes_cm_info *); -static int add_ref_cm_node(struct nes_cm_node *); -static int rem_ref_cm_node(struct nes_cm_core *, struct nes_cm_node *); + struct nes_vnic *, struct nes_cm_info *); static int mini_cm_del_listen(struct nes_cm_core *, struct nes_cm_listener *); -static struct sk_buff *form_cm_frame(struct sk_buff *, struct nes_cm_node *, - void *, u32, void *, u32, u8); -static struct sk_buff *get_free_pkt(struct nes_cm_node *cm_node); - static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *, - struct nes_vnic *, - struct ietf_mpa_frame *, - struct nes_cm_info *); + struct nes_vnic *, u16, void *, struct nes_cm_info *); +static int mini_cm_close(struct nes_cm_core *, struct nes_cm_node *); static int mini_cm_accept(struct nes_cm_core *, struct ietf_mpa_frame *, - struct nes_cm_node *); + struct nes_cm_node *); static int mini_cm_reject(struct nes_cm_core *, struct ietf_mpa_frame *, - struct nes_cm_node *); -static int mini_cm_close(struct nes_cm_core *, struct nes_cm_node *); -static int mini_cm_recv_pkt(struct nes_cm_core *, struct nes_vnic *, - struct sk_buff *); + struct nes_cm_node *); +static void mini_cm_recv_pkt(struct nes_cm_core *, struct nes_vnic *, + struct sk_buff *); static int mini_cm_dealloc_core(struct nes_cm_core *); static int mini_cm_get(struct nes_cm_core *); static int mini_cm_set(struct nes_cm_core *, u32, u32); + +static struct sk_buff *form_cm_frame(struct sk_buff *, struct nes_cm_node *, + void *, u32, void *, u32, u8); +static struct sk_buff *get_free_pkt(struct nes_cm_node *cm_node); +static int add_ref_cm_node(struct nes_cm_node *); +static int rem_ref_cm_node(struct nes_cm_core *, struct nes_cm_node *); + static int nes_cm_disconn_true(struct nes_qp *); static int nes_cm_post_event(struct nes_cm_event *event); static int nes_disconnect(struct nes_qp *nesqp, int abrupt); static void nes_disconnect_worker(struct work_struct *work); -static int send_ack(struct nes_cm_node *cm_node); + +static int send_mpa_request(struct nes_cm_node *, struct sk_buff *); +static int send_syn(struct nes_cm_node *, u32, struct sk_buff *); +static int send_reset(struct nes_cm_node *, struct sk_buff *); +static int send_ack(struct nes_cm_node *cm_node, struct sk_buff *skb); static int send_fin(struct nes_cm_node *cm_node, struct sk_buff *skb); +static void process_packet(struct nes_cm_node *, struct sk_buff *, + struct nes_cm_core *); + +static void active_open_err(struct nes_cm_node *, struct sk_buff *, int); +static void passive_open_err(struct nes_cm_node *, struct sk_buff *, int); +static void cleanup_retrans_entry(struct nes_cm_node *); +static void handle_rcv_mpa(struct nes_cm_node *, struct sk_buff *, + enum nes_cm_event_type); +static void free_retrans_entry(struct nes_cm_node *cm_node); +static int handle_tcp_options(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb, int optionsize, int passive); + +/* CM event handler functions */ +static void cm_event_connected(struct nes_cm_event *); +static void cm_event_connect_error(struct nes_cm_event *); +static void cm_event_reset(struct nes_cm_event *); +static void cm_event_mpa_req(struct nes_cm_event *); + +static void print_core(struct nes_cm_core *core); /* External CM API Interface */ /* instance of function pointers for client API */ @@ -158,11 +181,11 @@ static struct nes_cm_event *create_event(struct nes_cm_node *cm_node, event->cm_info.loc_port = cm_node->loc_port; event->cm_info.cm_id = cm_node->cm_id; - nes_debug(NES_DBG_CM, "Created event=%p, type=%u, dst_addr=%08x[%x]," - " src_addr=%08x[%x]\n", - event, type, - event->cm_info.loc_addr, event->cm_info.loc_port, - event->cm_info.rem_addr, event->cm_info.rem_port); + nes_debug(NES_DBG_CM, "cm_node=%p Created event=%p, type=%u, " + "dst_addr=%08x[%x], src_addr=%08x[%x]\n", + cm_node, event, type, event->cm_info.loc_addr, + event->cm_info.loc_port, event->cm_info.rem_addr, + event->cm_info.rem_port); nes_cm_post_event(event); return event; @@ -172,14 +195,11 @@ static struct nes_cm_event *create_event(struct nes_cm_node *cm_node, /** * send_mpa_request */ -static int send_mpa_request(struct nes_cm_node *cm_node) +static int send_mpa_request(struct nes_cm_node *cm_node, struct sk_buff *skb) { - struct sk_buff *skb; int ret; - - skb = get_free_pkt(cm_node); if (!skb) { - nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); + nes_debug(NES_DBG_CM, "skb set to NULL\n"); return -1; } @@ -188,9 +208,8 @@ static int send_mpa_request(struct nes_cm_node *cm_node) cm_node->mpa_frame_size, SET_ACK); ret = schedule_nes_timer(cm_node, skb, NES_TIMER_TYPE_SEND, 1, 0); - if (ret < 0) { + if (ret < 0) return ret; - } return 0; } @@ -229,46 +248,12 @@ static int parse_mpa(struct nes_cm_node *cm_node, u8 *buffer, u32 len) /** - * handle_exception_pkt - process an exception packet. - * We have been in a TSA state, and we have now received SW - * TCP/IP traffic should be a FIN request or IP pkt with options - */ -static int handle_exception_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb) -{ - int ret = 0; - struct tcphdr *tcph = tcp_hdr(skb); - - /* first check to see if this a FIN pkt */ - if (tcph->fin) { - /* we need to ACK the FIN request */ - send_ack(cm_node); - - /* check which side we are (client/server) and set next state accordingly */ - if (cm_node->tcp_cntxt.client) - cm_node->state = NES_CM_STATE_CLOSING; - else { - /* we are the server side */ - cm_node->state = NES_CM_STATE_CLOSE_WAIT; - /* since this is a self contained CM we don't wait for */ - /* an APP to close us, just send final FIN immediately */ - ret = send_fin(cm_node, NULL); - cm_node->state = NES_CM_STATE_LAST_ACK; - } - } else { - ret = -EINVAL; - } - - return ret; -} - - -/** * form_cm_frame - get a free packet and build empty frame Use * node info to build. */ -static struct sk_buff *form_cm_frame(struct sk_buff *skb, struct nes_cm_node *cm_node, - void *options, u32 optionsize, void *data, - u32 datasize, u8 flags) +static struct sk_buff *form_cm_frame(struct sk_buff *skb, + struct nes_cm_node *cm_node, void *options, u32 optionsize, + void *data, u32 datasize, u8 flags) { struct tcphdr *tcph; struct iphdr *iph; @@ -332,10 +317,12 @@ static struct sk_buff *form_cm_frame(struct sk_buff *skb, struct nes_cm_node *cm cm_node->tcp_cntxt.loc_seq_num++; tcph->syn = 1; } else - cm_node->tcp_cntxt.loc_seq_num += datasize; /* data (no headers) */ + cm_node->tcp_cntxt.loc_seq_num += datasize; - if (flags & SET_FIN) + if (flags & SET_FIN) { + cm_node->tcp_cntxt.loc_seq_num++; tcph->fin = 1; + } if (flags & SET_RST) tcph->rst = 1; @@ -389,7 +376,7 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, int close_when_complete) { unsigned long flags; - struct nes_cm_core *cm_core; + struct nes_cm_core *cm_core = cm_node->cm_core; struct nes_timer_entry *new_send; int ret = 0; u32 was_timer_set; @@ -411,7 +398,7 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, new_send->close_when_complete = close_when_complete; if (type == NES_TIMER_TYPE_CLOSE) { - new_send->timetosend += (HZ/2); /* TODO: decide on the correct value here */ + new_send->timetosend += (HZ/10); spin_lock_irqsave(&cm_node->recv_list_lock, flags); list_add_tail(&new_send->list, &cm_node->recv_list); spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); @@ -420,36 +407,28 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, if (type == NES_TIMER_TYPE_SEND) { new_send->seq_num = ntohl(tcp_hdr(skb)->seq); atomic_inc(&new_send->skb->users); + spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + cm_node->send_entry = new_send; + add_ref_cm_node(cm_node); + spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); + new_send->timetosend = jiffies + NES_RETRY_TIMEOUT; ret = nes_nic_cm_xmit(new_send->skb, cm_node->netdev); if (ret != NETDEV_TX_OK) { - nes_debug(NES_DBG_CM, "Error sending packet %p (jiffies = %lu)\n", - new_send, jiffies); + nes_debug(NES_DBG_CM, "Error sending packet %p " + "(jiffies = %lu)\n", new_send, jiffies); atomic_dec(&new_send->skb->users); new_send->timetosend = jiffies; } else { cm_packets_sent++; if (!send_retrans) { + cleanup_retrans_entry(cm_node); if (close_when_complete) - rem_ref_cm_node(cm_node->cm_core, cm_node); - dev_kfree_skb_any(new_send->skb); - kfree(new_send); + rem_ref_cm_node(cm_core, cm_node); return ret; } - new_send->timetosend = jiffies + NES_RETRY_TIMEOUT; } - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - list_add_tail(&new_send->list, &cm_node->retrans_list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - } - if (type == NES_TIMER_TYPE_RECV) { - new_send->seq_num = ntohl(tcp_hdr(skb)->seq); - new_send->timetosend = jiffies; - spin_lock_irqsave(&cm_node->recv_list_lock, flags); - list_add_tail(&new_send->list, &cm_node->recv_list); - spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); } - cm_core = cm_node->cm_core; was_timer_set = timer_pending(&cm_core->tcp_timer); @@ -476,23 +455,27 @@ static void nes_cm_timer_tick(unsigned long pass) struct list_head *list_node, *list_node_temp; struct nes_cm_core *cm_core = g_cm_core; struct nes_qp *nesqp; - struct sk_buff *skb; u32 settimer = 0; int ret = NETDEV_TX_OK; - int node_done; + enum nes_cm_node_state last_state; spin_lock_irqsave(&cm_core->ht_lock, flags); - list_for_each_safe(list_node, list_core_temp, &cm_core->connected_nodes) { + list_for_each_safe(list_node, list_core_temp, + &cm_core->connected_nodes) { cm_node = container_of(list_node, struct nes_cm_node, list); add_ref_cm_node(cm_node); spin_unlock_irqrestore(&cm_core->ht_lock, flags); spin_lock_irqsave(&cm_node->recv_list_lock, flags); - list_for_each_safe(list_core, list_node_temp, &cm_node->recv_list) { - recv_entry = container_of(list_core, struct nes_timer_entry, list); - if ((time_after(recv_entry->timetosend, jiffies)) && - (recv_entry->type == NES_TIMER_TYPE_CLOSE)) { - if (nexttimeout > recv_entry->timetosend || !settimer) { + list_for_each_safe(list_core, list_node_temp, + &cm_node->recv_list) { + recv_entry = container_of(list_core, + struct nes_timer_entry, list); + if (!recv_entry) + break; + if (time_after(recv_entry->timetosend, jiffies)) { + if (nexttimeout > recv_entry->timetosend || + !settimer) { nexttimeout = recv_entry->timetosend; settimer = 1; } @@ -501,157 +484,143 @@ static void nes_cm_timer_tick(unsigned long pass) list_del(&recv_entry->list); cm_id = cm_node->cm_id; spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); - if (recv_entry->type == NES_TIMER_TYPE_CLOSE) { - nesqp = (struct nes_qp *)recv_entry->skb; - spin_lock_irqsave(&nesqp->lock, qplockflags); - if (nesqp->cm_id) { - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, refcount = %d: " - "****** HIT A NES_TIMER_TYPE_CLOSE" - " with something to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id, - atomic_read(&nesqp->refcount)); - nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; - nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; - nesqp->ibqp_state = IB_QPS_ERR; - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_cm_disconn(nesqp); - } else { - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, refcount = %d:" - " ****** HIT A NES_TIMER_TYPE_CLOSE" - " with nothing to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id, - atomic_read(&nesqp->refcount)); - nes_rem_ref(&nesqp->ibqp); - } - if (cm_id) - cm_id->rem_ref(cm_id); + nesqp = (struct nes_qp *)recv_entry->skb; + spin_lock_irqsave(&nesqp->lock, qplockflags); + if (nesqp->cm_id) { + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, " + "refcount = %d: HIT A " + "NES_TIMER_TYPE_CLOSE with something " + "to do!!!\n", nesqp->hwqp.qp_id, cm_id, + atomic_read(&nesqp->refcount)); + nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; + nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; + nesqp->ibqp_state = IB_QPS_ERR; + spin_unlock_irqrestore(&nesqp->lock, + qplockflags); + nes_cm_disconn(nesqp); + } else { + spin_unlock_irqrestore(&nesqp->lock, + qplockflags); + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, " + "refcount = %d: HIT A " + "NES_TIMER_TYPE_CLOSE with nothing " + "to do!!!\n", nesqp->hwqp.qp_id, cm_id, + atomic_read(&nesqp->refcount)); } + if (cm_id) + cm_id->rem_ref(cm_id); + kfree(recv_entry); spin_lock_irqsave(&cm_node->recv_list_lock, flags); } spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - node_done = 0; - list_for_each_safe(list_core, list_node_temp, &cm_node->retrans_list) { - if (node_done) { - break; - } - send_entry = container_of(list_core, struct nes_timer_entry, list); + do { + send_entry = cm_node->send_entry; + if (!send_entry) + continue; if (time_after(send_entry->timetosend, jiffies)) { if (cm_node->state != NES_CM_STATE_TSA) { - if ((nexttimeout > send_entry->timetosend) || !settimer) { - nexttimeout = send_entry->timetosend; + if ((nexttimeout > + send_entry->timetosend) || + !settimer) { + nexttimeout = + send_entry->timetosend; settimer = 1; + continue; } - node_done = 1; - continue; } else { - list_del(&send_entry->list); - skb = send_entry->skb; - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - dev_kfree_skb_any(skb); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + free_retrans_entry(cm_node); continue; } } - if (send_entry->type == NES_TIMER_NODE_CLEANUP) { - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - continue; - } - if ((send_entry->seq_num < cm_node->tcp_cntxt.rem_ack_num) || - (cm_node->state == NES_CM_STATE_TSA) || - (cm_node->state == NES_CM_STATE_CLOSED)) { - skb = send_entry->skb; - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - kfree(send_entry); - dev_kfree_skb_any(skb); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + + if ((cm_node->state == NES_CM_STATE_TSA) || + (cm_node->state == NES_CM_STATE_CLOSED)) { + free_retrans_entry(cm_node); continue; } - if (!send_entry->retranscount || !send_entry->retrycount) { + if (!send_entry->retranscount || + !send_entry->retrycount) { cm_packets_dropped++; - skb = send_entry->skb; - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - dev_kfree_skb_any(skb); - kfree(send_entry); - if (cm_node->state == NES_CM_STATE_SYN_RCVD) { - /* this node never even generated an indication up to the cm */ + last_state = cm_node->state; + cm_node->state = NES_CM_STATE_CLOSED; + free_retrans_entry(cm_node); + spin_unlock_irqrestore( + &cm_node->retrans_list_lock, flags); + if (last_state == NES_CM_STATE_SYN_RCVD) rem_ref_cm_node(cm_core, cm_node); - } else { - cm_node->state = NES_CM_STATE_CLOSED; - create_event(cm_node, NES_CM_EVENT_ABORTED); - } - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + else + create_event(cm_node, + NES_CM_EVENT_ABORTED); + spin_lock_irqsave(&cm_node->retrans_list_lock, + flags); continue; } - /* this seems like the correct place, but leave send entry unprotected */ - /* spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); */ atomic_inc(&send_entry->skb->users); cm_packets_retrans++; - nes_debug(NES_DBG_CM, "Retransmitting send_entry %p for node %p," - " jiffies = %lu, time to send = %lu, retranscount = %u, " - "send_entry->seq_num = 0x%08X, cm_node->tcp_cntxt.rem_ack_num = 0x%08X\n", - send_entry, cm_node, jiffies, send_entry->timetosend, send_entry->retranscount, - send_entry->seq_num, cm_node->tcp_cntxt.rem_ack_num); - - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); + nes_debug(NES_DBG_CM, "Retransmitting send_entry %p " + "for node %p, jiffies = %lu, time to send = " + "%lu, retranscount = %u, send_entry->seq_num = " + "0x%08X, cm_node->tcp_cntxt.rem_ack_num = " + "0x%08X\n", send_entry, cm_node, jiffies, + send_entry->timetosend, + send_entry->retranscount, + send_entry->seq_num, + cm_node->tcp_cntxt.rem_ack_num); + + spin_unlock_irqrestore(&cm_node->retrans_list_lock, + flags); ret = nes_nic_cm_xmit(send_entry->skb, cm_node->netdev); + spin_lock_irqsave(&cm_node->retrans_list_lock, flags); if (ret != NETDEV_TX_OK) { + nes_debug(NES_DBG_CM, "rexmit failed for " + "node=%p\n", cm_node); cm_packets_bounced++; atomic_dec(&send_entry->skb->users); send_entry->retrycount--; nexttimeout = jiffies + NES_SHORT_TIME; settimer = 1; - node_done = 1; - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); continue; } else { cm_packets_sent++; } - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - list_del(&send_entry->list); - nes_debug(NES_DBG_CM, "Packet Sent: retrans count = %u, retry count = %u.\n", - send_entry->retranscount, send_entry->retrycount); + nes_debug(NES_DBG_CM, "Packet Sent: retrans count = " + "%u, retry count = %u.\n", + send_entry->retranscount, + send_entry->retrycount); if (send_entry->send_retrans) { send_entry->retranscount--; - send_entry->timetosend = jiffies + NES_RETRY_TIMEOUT; - if (nexttimeout > send_entry->timetosend || !settimer) { + send_entry->timetosend = jiffies + + NES_RETRY_TIMEOUT; + if (nexttimeout > send_entry->timetosend || + !settimer) { nexttimeout = send_entry->timetosend; settimer = 1; } - list_add(&send_entry->list, &cm_node->retrans_list); - continue; } else { int close_when_complete; - skb = send_entry->skb; - close_when_complete = send_entry->close_when_complete; - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - if (close_when_complete) { - BUG_ON(atomic_read(&cm_node->ref_count) == 1); - rem_ref_cm_node(cm_core, cm_node); - } - dev_kfree_skb_any(skb); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - continue; + close_when_complete = + send_entry->close_when_complete; + nes_debug(NES_DBG_CM, "cm_node=%p state=%d\n", + cm_node, cm_node->state); + free_retrans_entry(cm_node); + if (close_when_complete) + rem_ref_cm_node(cm_node->cm_core, + cm_node); } - } - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - - rem_ref_cm_node(cm_core, cm_node); + } while (0); + spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); + rem_ref_cm_node(cm_node->cm_core, cm_node); spin_lock_irqsave(&cm_core->ht_lock, flags); - if (ret != NETDEV_TX_OK) + if (ret != NETDEV_TX_OK) { + nes_debug(NES_DBG_CM, "rexmit failed for cm_node=%p\n", + cm_node); break; + } } spin_unlock_irqrestore(&cm_core->ht_lock, flags); @@ -667,14 +636,14 @@ static void nes_cm_timer_tick(unsigned long pass) /** * send_syn */ -static int send_syn(struct nes_cm_node *cm_node, u32 sendack) +static int send_syn(struct nes_cm_node *cm_node, u32 sendack, + struct sk_buff *skb) { int ret; int flags = SET_SYN; - struct sk_buff *skb; char optionsbuffer[sizeof(struct option_mss) + - sizeof(struct option_windowscale) + - sizeof(struct option_base) + 1]; + sizeof(struct option_windowscale) + sizeof(struct option_base) + + TCP_OPTIONS_PADDING]; int optionssize = 0; /* Sending MSS option */ @@ -695,8 +664,7 @@ static int send_syn(struct nes_cm_node *cm_node, u32 sendack) options->as_windowscale.shiftcount = cm_node->tcp_cntxt.rcv_wscale; optionssize += sizeof(struct option_windowscale); - if (sendack && !(NES_DRV_OPT_SUPRESS_OPTION_BC & nes_drv_opt) - ) { + if (sendack && !(NES_DRV_OPT_SUPRESS_OPTION_BC & nes_drv_opt)) { options = (union all_known_options *)&optionsbuffer[optionssize]; options->as_base.optionnum = OPTION_NUMBER_WRITE0; options->as_base.length = sizeof(struct option_base); @@ -714,7 +682,8 @@ static int send_syn(struct nes_cm_node *cm_node, u32 sendack) options->as_end = OPTION_NUMBER_END; optionssize += 1; - skb = get_free_pkt(cm_node); + if (!skb) + skb = get_free_pkt(cm_node); if (!skb) { nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); return -1; @@ -733,18 +702,18 @@ static int send_syn(struct nes_cm_node *cm_node, u32 sendack) /** * send_reset */ -static int send_reset(struct nes_cm_node *cm_node) +static int send_reset(struct nes_cm_node *cm_node, struct sk_buff *skb) { int ret; - struct sk_buff *skb = get_free_pkt(cm_node); int flags = SET_RST | SET_ACK; + if (!skb) + skb = get_free_pkt(cm_node); if (!skb) { nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); return -1; } - add_ref_cm_node(cm_node); form_cm_frame(skb, cm_node, NULL, 0, NULL, 0, flags); ret = schedule_nes_timer(cm_node, skb, NES_TIMER_TYPE_SEND, 0, 1); @@ -755,10 +724,12 @@ static int send_reset(struct nes_cm_node *cm_node) /** * send_ack */ -static int send_ack(struct nes_cm_node *cm_node) +static int send_ack(struct nes_cm_node *cm_node, struct sk_buff *skb) { int ret; - struct sk_buff *skb = get_free_pkt(cm_node); + + if (!skb) + skb = get_free_pkt(cm_node); if (!skb) { nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); @@ -922,7 +893,8 @@ static int add_hte_node(struct nes_cm_core *cm_core, struct nes_cm_node *cm_node if (!cm_node || !cm_core) return -EINVAL; - nes_debug(NES_DBG_CM, "Adding Node to Active Connection HT\n"); + nes_debug(NES_DBG_CM, "Adding Node %p to Active Connection HT\n", + cm_node); /* first, make an index into our hash table */ hashkey = make_hashkey(cm_node->loc_port, cm_node->loc_addr, @@ -946,10 +918,35 @@ static int add_hte_node(struct nes_cm_core *cm_core, struct nes_cm_node *cm_node * mini_cm_dec_refcnt_listen */ static int mini_cm_dec_refcnt_listen(struct nes_cm_core *cm_core, - struct nes_cm_listener *listener, int free_hanging_nodes) + struct nes_cm_listener *listener, int free_hanging_nodes) { int ret = 1; unsigned long flags; + struct list_head *list_pos = NULL; + struct list_head *list_temp = NULL; + struct nes_cm_node *cm_node = NULL; + + nes_debug(NES_DBG_CM, "attempting listener= %p free_nodes= %d, " + "refcnt=%d\n", listener, free_hanging_nodes, + atomic_read(&listener->ref_count)); + /* free non-accelerated child nodes for this listener */ + if (free_hanging_nodes) { + spin_lock_irqsave(&cm_core->ht_lock, flags); + list_for_each_safe(list_pos, list_temp, + &g_cm_core->connected_nodes) { + cm_node = container_of(list_pos, struct nes_cm_node, + list); + if ((cm_node->listener == listener) && + (!cm_node->accelerated)) { + cleanup_retrans_entry(cm_node); + spin_unlock_irqrestore(&cm_core->ht_lock, + flags); + send_reset(cm_node, NULL); + spin_lock_irqsave(&cm_core->ht_lock, flags); + } + } + spin_unlock_irqrestore(&cm_core->ht_lock, flags); + } spin_lock_irqsave(&cm_core->listen_list_lock, flags); if (!atomic_dec_return(&listener->ref_count)) { list_del(&listener->list); @@ -1067,18 +1064,18 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, cm_node->loc_port = cm_info->loc_port; cm_node->rem_port = cm_info->rem_port; cm_node->send_write0 = send_first; - nes_debug(NES_DBG_CM, "Make node addresses : loc = " NIPQUAD_FMT ":%x, rem = " NIPQUAD_FMT ":%x\n", - HIPQUAD(cm_node->loc_addr), cm_node->loc_port, - HIPQUAD(cm_node->rem_addr), cm_node->rem_port); + nes_debug(NES_DBG_CM, "Make node addresses : loc = " NIPQUAD_FMT + ":%x, rem = " NIPQUAD_FMT ":%x\n", + HIPQUAD(cm_node->loc_addr), cm_node->loc_port, + HIPQUAD(cm_node->rem_addr), cm_node->rem_port); cm_node->listener = listener; cm_node->netdev = nesvnic->netdev; cm_node->cm_id = cm_info->cm_id; memcpy(cm_node->loc_mac, nesvnic->netdev->dev_addr, ETH_ALEN); - nes_debug(NES_DBG_CM, "listener=%p, cm_id=%p\n", - cm_node->listener, cm_node->cm_id); + nes_debug(NES_DBG_CM, "listener=%p, cm_id=%p\n", cm_node->listener, + cm_node->cm_id); - INIT_LIST_HEAD(&cm_node->retrans_list); spin_lock_init(&cm_node->retrans_list_lock); INIT_LIST_HEAD(&cm_node->recv_list); spin_lock_init(&cm_node->recv_list_lock); @@ -1142,10 +1139,9 @@ static int add_ref_cm_node(struct nes_cm_node *cm_node) * rem_ref_cm_node - destroy an instance of a cm node */ static int rem_ref_cm_node(struct nes_cm_core *cm_core, - struct nes_cm_node *cm_node) + struct nes_cm_node *cm_node) { unsigned long flags, qplockflags; - struct nes_timer_entry *send_entry; struct nes_timer_entry *recv_entry; struct iw_cm_id *cm_id; struct list_head *list_core, *list_node_temp; @@ -1169,48 +1165,33 @@ static int rem_ref_cm_node(struct nes_cm_core *cm_core, atomic_dec(&cm_node->listener->pend_accepts_cnt); BUG_ON(atomic_read(&cm_node->listener->pend_accepts_cnt) < 0); } - - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - list_for_each_safe(list_core, list_node_temp, &cm_node->retrans_list) { - send_entry = container_of(list_core, struct nes_timer_entry, list); - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - dev_kfree_skb_any(send_entry->skb); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - continue; - } - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - + BUG_ON(cm_node->send_entry); spin_lock_irqsave(&cm_node->recv_list_lock, flags); list_for_each_safe(list_core, list_node_temp, &cm_node->recv_list) { - recv_entry = container_of(list_core, struct nes_timer_entry, list); + recv_entry = container_of(list_core, struct nes_timer_entry, + list); list_del(&recv_entry->list); cm_id = cm_node->cm_id; spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); - if (recv_entry->type == NES_TIMER_TYPE_CLOSE) { - nesqp = (struct nes_qp *)recv_entry->skb; - spin_lock_irqsave(&nesqp->lock, qplockflags); - if (nesqp->cm_id) { - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: ****** HIT A NES_TIMER_TYPE_CLOSE" - " with something to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id); - nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; - nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; - nesqp->ibqp_state = IB_QPS_ERR; - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_cm_disconn(nesqp); - } else { - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: ****** HIT A NES_TIMER_TYPE_CLOSE" - " with nothing to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id); - nes_rem_ref(&nesqp->ibqp); - } - cm_id->rem_ref(cm_id); - } else if (recv_entry->type == NES_TIMER_TYPE_RECV) { - dev_kfree_skb_any(recv_entry->skb); + nesqp = (struct nes_qp *)recv_entry->skb; + spin_lock_irqsave(&nesqp->lock, qplockflags); + if (nesqp->cm_id) { + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: HIT A " + "NES_TIMER_TYPE_CLOSE with something to do!\n", + nesqp->hwqp.qp_id, cm_id); + nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; + nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; + nesqp->ibqp_state = IB_QPS_ERR; + spin_unlock_irqrestore(&nesqp->lock, qplockflags); + nes_cm_disconn(nesqp); + } else { + spin_unlock_irqrestore(&nesqp->lock, qplockflags); + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: HIT A " + "NES_TIMER_TYPE_CLOSE with nothing to do!\n", + nesqp->hwqp.qp_id, cm_id); } + cm_id->rem_ref(cm_id); + kfree(recv_entry); spin_lock_irqsave(&cm_node->recv_list_lock, flags); } @@ -1221,23 +1202,31 @@ static int rem_ref_cm_node(struct nes_cm_core *cm_core, } else { if (cm_node->apbvt_set && cm_node->nesvnic) { nes_manage_apbvt(cm_node->nesvnic, cm_node->loc_port, - PCI_FUNC(cm_node->nesvnic->nesdev->pcidev->devfn), - NES_MANAGE_APBVT_DEL); + PCI_FUNC( + cm_node->nesvnic->nesdev->pcidev->devfn), + NES_MANAGE_APBVT_DEL); } } - kfree(cm_node); atomic_dec(&cm_core->node_cnt); atomic_inc(&cm_nodes_destroyed); + nesqp = cm_node->nesqp; + if (nesqp) { + nesqp->cm_node = NULL; + nes_rem_ref(&nesqp->ibqp); + cm_node->nesqp = NULL; + } + cm_node->freed = 1; + kfree(cm_node); return 0; } - /** * process_options */ -static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 optionsize, u32 syn_packet) +static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, + u32 optionsize, u32 syn_packet) { u32 tmp; u32 offset = 0; @@ -1247,35 +1236,37 @@ static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 opti while (offset < optionsize) { all_options = (union all_known_options *)(optionsloc + offset); switch (all_options->as_base.optionnum) { - case OPTION_NUMBER_END: - offset = optionsize; - break; - case OPTION_NUMBER_NONE: - offset += 1; - continue; - case OPTION_NUMBER_MSS: - nes_debug(NES_DBG_CM, "%s: MSS Length: %d Offset: %d Size: %d\n", - __func__, - all_options->as_mss.length, offset, optionsize); - got_mss_option = 1; - if (all_options->as_mss.length != 4) { - return 1; - } else { - tmp = ntohs(all_options->as_mss.mss); - if (tmp > 0 && tmp < cm_node->tcp_cntxt.mss) - cm_node->tcp_cntxt.mss = tmp; - } - break; - case OPTION_NUMBER_WINDOW_SCALE: - cm_node->tcp_cntxt.snd_wscale = all_options->as_windowscale.shiftcount; - break; - case OPTION_NUMBER_WRITE0: - cm_node->send_write0 = 1; - break; - default: - nes_debug(NES_DBG_CM, "TCP Option not understood: %x\n", - all_options->as_base.optionnum); - break; + case OPTION_NUMBER_END: + offset = optionsize; + break; + case OPTION_NUMBER_NONE: + offset += 1; + continue; + case OPTION_NUMBER_MSS: + nes_debug(NES_DBG_CM, "%s: MSS Length: %d Offset: %d " + "Size: %d\n", __func__, + all_options->as_mss.length, offset, optionsize); + got_mss_option = 1; + if (all_options->as_mss.length != 4) { + return 1; + } else { + tmp = ntohs(all_options->as_mss.mss); + if (tmp > 0 && tmp < + cm_node->tcp_cntxt.mss) + cm_node->tcp_cntxt.mss = tmp; + } + break; + case OPTION_NUMBER_WINDOW_SCALE: + cm_node->tcp_cntxt.snd_wscale = + all_options->as_windowscale.shiftcount; + break; + case OPTION_NUMBER_WRITE0: + cm_node->send_write0 = 1; + break; + default: + nes_debug(NES_DBG_CM, "TCP Option not understood: %x\n", + all_options->as_base.optionnum); + break; } offset += all_options->as_base.length; } @@ -1284,300 +1275,491 @@ static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 opti return 0; } +inline void drop_packet(struct sk_buff *skb) +{ + atomic_inc(&cm_accel_dropped_pkts); + dev_kfree_skb_any(skb); +} -/** - * process_packet - */ -static int process_packet(struct nes_cm_node *cm_node, struct sk_buff *skb, - struct nes_cm_core *cm_core) +static void handle_fin_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) { - int optionsize; - int datasize; - int ret = 0; - struct tcphdr *tcph = tcp_hdr(skb); - u32 inc_sequence; - if (cm_node->state == NES_CM_STATE_SYN_SENT && tcph->syn) { - inc_sequence = ntohl(tcph->seq); - cm_node->tcp_cntxt.rcv_nxt = inc_sequence; + atomic_inc(&cm_resets_recvd); + nes_debug(NES_DBG_CM, "Received FIN, cm_node = %p, state = %u. " + "refcnt=%d\n", cm_node, cm_node->state, + atomic_read(&cm_node->ref_count)); + cm_node->tcp_cntxt.rcv_nxt++; + cleanup_retrans_entry(cm_node); + switch (cm_node->state) { + case NES_CM_STATE_SYN_RCVD: + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_MPAREQ_SENT: + cm_node->state = NES_CM_STATE_LAST_ACK; + send_fin(cm_node, skb); + break; + case NES_CM_STATE_FIN_WAIT1: + cm_node->state = NES_CM_STATE_CLOSING; + send_ack(cm_node, skb); + break; + case NES_CM_STATE_FIN_WAIT2: + cm_node->state = NES_CM_STATE_TIME_WAIT; + send_ack(cm_node, skb); + cm_node->state = NES_CM_STATE_CLOSED; + break; + case NES_CM_STATE_TSA: + default: + nes_debug(NES_DBG_CM, "Error Rcvd FIN for node-%p state = %d\n", + cm_node, cm_node->state); + drop_packet(skb); + break; } +} - if ((!tcph) || (cm_node->state == NES_CM_STATE_TSA)) { - BUG_ON(!tcph); - atomic_inc(&cm_accel_dropped_pkts); - return -1; - } - if (tcph->rst) { - atomic_inc(&cm_resets_recvd); - nes_debug(NES_DBG_CM, "Received Reset, cm_node = %p, state = %u. refcnt=%d\n", - cm_node, cm_node->state, atomic_read(&cm_node->ref_count)); - switch (cm_node->state) { - case NES_CM_STATE_LISTENING: - rem_ref_cm_node(cm_core, cm_node); - break; - case NES_CM_STATE_TSA: - case NES_CM_STATE_CLOSED: - break; - case NES_CM_STATE_SYN_RCVD: - nes_debug(NES_DBG_CM, "Received a reset for local 0x%08X:%04X," - " remote 0x%08X:%04X, node state = %u\n", - cm_node->loc_addr, cm_node->loc_port, - cm_node->rem_addr, cm_node->rem_port, - cm_node->state); - rem_ref_cm_node(cm_core, cm_node); - break; - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_MPAREQ_SENT: - default: - nes_debug(NES_DBG_CM, "Received a reset for local 0x%08X:%04X," - " remote 0x%08X:%04X, node state = %u refcnt=%d\n", - cm_node->loc_addr, cm_node->loc_port, - cm_node->rem_addr, cm_node->rem_port, - cm_node->state, atomic_read(&cm_node->ref_count)); - /* create event */ - cm_node->state = NES_CM_STATE_CLOSED; +static void handle_rst_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ - create_event(cm_node, NES_CM_EVENT_ABORTED); - break; + int reset = 0; /* whether to send reset in case of err.. */ + atomic_inc(&cm_resets_recvd); + nes_debug(NES_DBG_CM, "Received Reset, cm_node = %p, state = %u." + " refcnt=%d\n", cm_node, cm_node->state, + atomic_read(&cm_node->ref_count)); + cleanup_retrans_entry(cm_node); + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_MPAREQ_SENT: + nes_debug(NES_DBG_CM, "%s[%u] create abort for cm_node=%p " + "listener=%p state=%d\n", __func__, __LINE__, cm_node, + cm_node->listener, cm_node->state); + active_open_err(cm_node, skb, reset); + break; + /* For PASSIVE open states, remove the cm_node event */ + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_SYN_RCVD: + case NES_CM_STATE_LISTENING: + nes_debug(NES_DBG_CM, "Bad state %s[%u]\n", __func__, __LINE__); + passive_open_err(cm_node, skb, reset); + break; + case NES_CM_STATE_TSA: + default: + break; + } +} +static void handle_rcv_mpa(struct nes_cm_node *cm_node, struct sk_buff *skb, + enum nes_cm_event_type type) +{ + + int ret; + int datasize = skb->len; + u8 *dataloc = skb->data; + ret = parse_mpa(cm_node, dataloc, datasize); + if (ret < 0) { + nes_debug(NES_DBG_CM, "didn't like MPA Request\n"); + if (type == NES_CM_EVENT_CONNECTED) { + nes_debug(NES_DBG_CM, "%s[%u] create abort for " + "cm_node=%p listener=%p state=%d\n", __func__, + __LINE__, cm_node, cm_node->listener, + cm_node->state); + active_open_err(cm_node, skb, 1); + } else { + passive_open_err(cm_node, skb, 1); } - return -1; + } else { + cleanup_retrans_entry(cm_node); + dev_kfree_skb_any(skb); + if (type == NES_CM_EVENT_CONNECTED) + cm_node->state = NES_CM_STATE_TSA; + create_event(cm_node, type); + + } + return ; +} + +static void indicate_pkt_err(struct nes_cm_node *cm_node, struct sk_buff *skb) +{ + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_MPAREQ_SENT: + nes_debug(NES_DBG_CM, "%s[%u] create abort for cm_node=%p " + "listener=%p state=%d\n", __func__, __LINE__, cm_node, + cm_node->listener, cm_node->state); + active_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_SYN_RCVD: + passive_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_TSA: + default: + drop_packet(skb); } +} + +static int check_syn(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb) +{ + int err; + + err = ((ntohl(tcph->ack_seq) == cm_node->tcp_cntxt.loc_seq_num))? 0 : 1; + if (err) + active_open_err(cm_node, skb, 1); + + return err; +} + +static int check_seq(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb) +{ + int err = 0; + u32 seq; + u32 ack_seq; + u32 loc_seq_num = cm_node->tcp_cntxt.loc_seq_num; + u32 rcv_nxt = cm_node->tcp_cntxt.rcv_nxt; + u32 rcv_wnd; + seq = ntohl(tcph->seq); + ack_seq = ntohl(tcph->ack_seq); + rcv_wnd = cm_node->tcp_cntxt.rcv_wnd; + if (ack_seq != loc_seq_num) + err = 1; + else if ((seq + rcv_wnd) < rcv_nxt) + err = 1; + if (err) { + nes_debug(NES_DBG_CM, "%s[%u] create abort for cm_node=%p " + "listener=%p state=%d\n", __func__, __LINE__, cm_node, + cm_node->listener, cm_node->state); + indicate_pkt_err(cm_node, skb); + nes_debug(NES_DBG_CM, "seq ERROR cm_node =%p seq=0x%08X " + "rcv_nxt=0x%08X rcv_wnd=0x%x\n", cm_node, seq, rcv_nxt, + rcv_wnd); + } + return err; +} + +/* + * handle_syn_pkt() is for Passive node. The syn packet is received when a node + * is created with a listener or it may comein as rexmitted packet which in + * that case will be just dropped. + */ + +static void handle_syn_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ + int ret; + u32 inc_sequence; + int optionsize; optionsize = (tcph->doff << 2) - sizeof(struct tcphdr); + skb_pull(skb, tcph->doff << 2); + inc_sequence = ntohl(tcph->seq); - skb_pull(skb, ip_hdr(skb)->ihl << 2); + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_MPAREQ_SENT: + /* Rcvd syn on active open connection*/ + active_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_LISTENING: + /* Passive OPEN */ + cm_node->accept_pend = 1; + atomic_inc(&cm_node->listener->pend_accepts_cnt); + if (atomic_read(&cm_node->listener->pend_accepts_cnt) > + cm_node->listener->backlog) { + nes_debug(NES_DBG_CM, "drop syn due to backlog " + "pressure \n"); + cm_backlog_drops++; + passive_open_err(cm_node, skb, 0); + break; + } + ret = handle_tcp_options(cm_node, tcph, skb, optionsize, + 1); + if (ret) { + passive_open_err(cm_node, skb, 0); + /* drop pkt */ + break; + } + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + 1; + BUG_ON(cm_node->send_entry); + cm_node->state = NES_CM_STATE_SYN_RCVD; + send_syn(cm_node, 1, skb); + break; + case NES_CM_STATE_TSA: + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_MPAREQ_RCVD: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_CLOSING: + case NES_CM_STATE_UNKNOWN: + case NES_CM_STATE_CLOSED: + default: + drop_packet(skb); + break; + } +} + +static void handle_synack_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ + + int ret; + u32 inc_sequence; + int optionsize; + + optionsize = (tcph->doff << 2) - sizeof(struct tcphdr); skb_pull(skb, tcph->doff << 2); + inc_sequence = ntohl(tcph->seq); + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + /* active open */ + if (check_syn(cm_node, tcph, skb)) + return; + cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); + /* setup options */ + ret = handle_tcp_options(cm_node, tcph, skb, optionsize, 0); + if (ret) { + nes_debug(NES_DBG_CM, "cm_node=%p tcp_options failed\n", + cm_node); + break; + } + cleanup_retrans_entry(cm_node); + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + 1; + send_mpa_request(cm_node, skb); + cm_node->state = NES_CM_STATE_MPAREQ_SENT; + break; + case NES_CM_STATE_MPAREQ_RCVD: + /* passive open, so should not be here */ + passive_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_TSA: + case NES_CM_STATE_CLOSING: + case NES_CM_STATE_UNKNOWN: + case NES_CM_STATE_CLOSED: + case NES_CM_STATE_MPAREQ_SENT: + default: + drop_packet(skb); + break; + } +} - datasize = skb->len; +static void handle_ack_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ + int datasize = 0; + u32 inc_sequence; + u32 rem_seq_ack; + u32 rem_seq; + if (check_seq(cm_node, tcph, skb)) + return; + + skb_pull(skb, tcph->doff << 2); inc_sequence = ntohl(tcph->seq); - nes_debug(NES_DBG_CM, "datasize = %u, sequence = 0x%08X, ack_seq = 0x%08X," - " rcv_nxt = 0x%08X Flags: %s %s.\n", - datasize, inc_sequence, ntohl(tcph->ack_seq), - cm_node->tcp_cntxt.rcv_nxt, (tcph->syn ? "SYN":""), - (tcph->ack ? "ACK":"")); - - if (!tcph->syn && (inc_sequence != cm_node->tcp_cntxt.rcv_nxt) - ) { - nes_debug(NES_DBG_CM, "dropping packet, datasize = %u, sequence = 0x%08X," - " ack_seq = 0x%08X, rcv_nxt = 0x%08X Flags: %s.\n", - datasize, inc_sequence, ntohl(tcph->ack_seq), - cm_node->tcp_cntxt.rcv_nxt, (tcph->ack ? "ACK":"")); - if (cm_node->state == NES_CM_STATE_LISTENING) { - rem_ref_cm_node(cm_core, cm_node); + rem_seq = ntohl(tcph->seq); + rem_seq_ack = ntohl(tcph->ack_seq); + datasize = skb->len; + + switch (cm_node->state) { + case NES_CM_STATE_SYN_RCVD: + /* Passive OPEN */ + cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); + cm_node->state = NES_CM_STATE_ESTABLISHED; + if (datasize) { + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; + cm_node->state = NES_CM_STATE_MPAREQ_RCVD; + handle_rcv_mpa(cm_node, skb, NES_CM_EVENT_MPA_REQ); + } else { /* rcvd ACK only */ + dev_kfree_skb_any(skb); + cleanup_retrans_entry(cm_node); + } + break; + case NES_CM_STATE_ESTABLISHED: + /* Passive OPEN */ + /* We expect mpa frame to be received only */ + if (datasize) { + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; + cm_node->state = NES_CM_STATE_MPAREQ_RCVD; + handle_rcv_mpa(cm_node, skb, + NES_CM_EVENT_MPA_REQ); + } else + drop_packet(skb); + break; + case NES_CM_STATE_MPAREQ_SENT: + cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); + if (datasize) { + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; + handle_rcv_mpa(cm_node, skb, NES_CM_EVENT_CONNECTED); + } else { /* Could be just an ack pkt.. */ + cleanup_retrans_entry(cm_node); + dev_kfree_skb_any(skb); } - return -1; + break; + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_TSA: + case NES_CM_STATE_CLOSED: + case NES_CM_STATE_MPAREQ_RCVD: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_CLOSING: + case NES_CM_STATE_UNKNOWN: + default: + drop_packet(skb); + break; } +} - cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; +static int handle_tcp_options(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb, int optionsize, int passive) +{ + u8 *optionsloc = (u8 *)&tcph[1]; if (optionsize) { - u8 *optionsloc = (u8 *)&tcph[1]; - if (process_options(cm_node, optionsloc, optionsize, (u32)tcph->syn)) { - nes_debug(NES_DBG_CM, "%s: Node %p, Sending RESET\n", __func__, cm_node); - send_reset(cm_node); - if (cm_node->state != NES_CM_STATE_SYN_SENT) - rem_ref_cm_node(cm_core, cm_node); - return 0; + if (process_options(cm_node, optionsloc, optionsize, + (u32)tcph->syn)) { + nes_debug(NES_DBG_CM, "%s: Node %p, Sending RESET\n", + __func__, cm_node); + if (passive) + passive_open_err(cm_node, skb, 0); + else + active_open_err(cm_node, skb, 0); + return 1; } - } else if (tcph->syn) - cm_node->tcp_cntxt.mss = NES_CM_DEFAULT_MSS; + } cm_node->tcp_cntxt.snd_wnd = ntohs(tcph->window) << cm_node->tcp_cntxt.snd_wscale; - if (cm_node->tcp_cntxt.snd_wnd > cm_node->tcp_cntxt.max_snd_wnd) { + if (cm_node->tcp_cntxt.snd_wnd > cm_node->tcp_cntxt.max_snd_wnd) cm_node->tcp_cntxt.max_snd_wnd = cm_node->tcp_cntxt.snd_wnd; - } + return 0; +} - if (tcph->ack) { - cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); - switch (cm_node->state) { - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - /* read and stash current sequence number */ - if (cm_node->tcp_cntxt.rem_ack_num != cm_node->tcp_cntxt.loc_seq_num) { - nes_debug(NES_DBG_CM, "ERROR - cm_node->tcp_cntxt.rem_ack_num !=" - " cm_node->tcp_cntxt.loc_seq_num\n"); - send_reset(cm_node); - return 0; - } - if (cm_node->state == NES_CM_STATE_SYN_SENT) - cm_node->state = NES_CM_STATE_ONE_SIDE_ESTABLISHED; - else { - cm_node->state = NES_CM_STATE_ESTABLISHED; - } - break; - case NES_CM_STATE_LAST_ACK: - cm_node->state = NES_CM_STATE_CLOSED; - break; - case NES_CM_STATE_FIN_WAIT1: - cm_node->state = NES_CM_STATE_FIN_WAIT2; - break; - case NES_CM_STATE_CLOSING: - cm_node->state = NES_CM_STATE_TIME_WAIT; - /* need to schedule this to happen in 2MSL timeouts */ - cm_node->state = NES_CM_STATE_CLOSED; - break; - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_MPAREQ_SENT: - case NES_CM_STATE_CLOSE_WAIT: - case NES_CM_STATE_TIME_WAIT: - case NES_CM_STATE_CLOSED: - break; - case NES_CM_STATE_LISTENING: - nes_debug(NES_DBG_CM, "Received an ACK on a listening port (SYN %d)\n", tcph->syn); - cm_node->tcp_cntxt.loc_seq_num = ntohl(tcph->ack_seq); - send_reset(cm_node); - /* send_reset bumps refcount, this should have been a new node */ - rem_ref_cm_node(cm_core, cm_node); - return -1; - break; - case NES_CM_STATE_TSA: - nes_debug(NES_DBG_CM, "Received a packet with the ack bit set while in TSA state\n"); - break; - case NES_CM_STATE_UNKNOWN: - case NES_CM_STATE_INITED: - case NES_CM_STATE_ACCEPTING: - case NES_CM_STATE_FIN_WAIT2: - default: - nes_debug(NES_DBG_CM, "Received ack from unknown state: %x\n", - cm_node->state); - send_reset(cm_node); - break; - } - } +/* + * active_open_err() will send reset() if flag set.. + * It will also send ABORT event. + */ - if (tcph->syn) { - if (cm_node->state == NES_CM_STATE_LISTENING) { - /* do not exceed backlog */ - atomic_inc(&cm_node->listener->pend_accepts_cnt); - if (atomic_read(&cm_node->listener->pend_accepts_cnt) > - cm_node->listener->backlog) { - nes_debug(NES_DBG_CM, "drop syn due to backlog pressure \n"); - cm_backlog_drops++; - atomic_dec(&cm_node->listener->pend_accepts_cnt); - rem_ref_cm_node(cm_core, cm_node); - return 0; - } - cm_node->accept_pend = 1; +static void active_open_err(struct nes_cm_node *cm_node, struct sk_buff *skb, + int reset) +{ + cleanup_retrans_entry(cm_node); + if (reset) { + nes_debug(NES_DBG_CM, "ERROR active err called for cm_node=%p, " + "state=%d\n", cm_node, cm_node->state); + add_ref_cm_node(cm_node); + send_reset(cm_node, skb); + } else + dev_kfree_skb_any(skb); - } - if (datasize == 0) - cm_node->tcp_cntxt.rcv_nxt ++; + cm_node->state = NES_CM_STATE_CLOSED; + create_event(cm_node, NES_CM_EVENT_ABORTED); +} - if (cm_node->state == NES_CM_STATE_LISTENING) { - cm_node->state = NES_CM_STATE_SYN_RCVD; - send_syn(cm_node, 1); - } - if (cm_node->state == NES_CM_STATE_ONE_SIDE_ESTABLISHED) { - cm_node->state = NES_CM_STATE_ESTABLISHED; - /* send final handshake ACK */ - ret = send_ack(cm_node); - if (ret < 0) - return ret; +/* + * passive_open_err() will either do a reset() or will free up the skb and + * remove the cm_node. + */ - cm_node->state = NES_CM_STATE_MPAREQ_SENT; - ret = send_mpa_request(cm_node); - if (ret < 0) - return ret; - } +static void passive_open_err(struct nes_cm_node *cm_node, struct sk_buff *skb, + int reset) +{ + cleanup_retrans_entry(cm_node); + cm_node->state = NES_CM_STATE_CLOSED; + if (reset) { + nes_debug(NES_DBG_CM, "passive_open_err sending RST for " + "cm_node=%p state =%d\n", cm_node, cm_node->state); + send_reset(cm_node, skb); + } else { + dev_kfree_skb_any(skb); + rem_ref_cm_node(cm_node->cm_core, cm_node); } +} - if (tcph->fin) { - cm_node->tcp_cntxt.rcv_nxt++; - switch (cm_node->state) { - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_ACCEPTING: - case NES_CM_STATE_MPAREQ_SENT: - cm_node->state = NES_CM_STATE_CLOSE_WAIT; - cm_node->state = NES_CM_STATE_LAST_ACK; - ret = send_fin(cm_node, NULL); - break; - case NES_CM_STATE_FIN_WAIT1: - cm_node->state = NES_CM_STATE_CLOSING; - ret = send_ack(cm_node); - break; - case NES_CM_STATE_FIN_WAIT2: - cm_node->state = NES_CM_STATE_TIME_WAIT; - cm_node->tcp_cntxt.loc_seq_num ++; - ret = send_ack(cm_node); - /* need to schedule this to happen in 2MSL timeouts */ - cm_node->state = NES_CM_STATE_CLOSED; - break; - case NES_CM_STATE_CLOSE_WAIT: - case NES_CM_STATE_LAST_ACK: - case NES_CM_STATE_CLOSING: - case NES_CM_STATE_TSA: - default: - nes_debug(NES_DBG_CM, "Received a fin while in %x state\n", - cm_node->state); - ret = -EINVAL; - break; - } +/* + * free_retrans_entry() routines assumes that the retrans_list_lock has + * been acquired before calling. + */ +static void free_retrans_entry(struct nes_cm_node *cm_node) +{ + struct nes_timer_entry *send_entry; + send_entry = cm_node->send_entry; + if (send_entry) { + cm_node->send_entry = NULL; + dev_kfree_skb_any(send_entry->skb); + kfree(send_entry); + rem_ref_cm_node(cm_node->cm_core, cm_node); } +} - if (datasize) { - u8 *dataloc = skb->data; - /* figure out what state we are in and handle transition to next state */ - switch (cm_node->state) { - case NES_CM_STATE_LISTENING: - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - case NES_CM_STATE_FIN_WAIT1: - case NES_CM_STATE_FIN_WAIT2: - case NES_CM_STATE_CLOSE_WAIT: - case NES_CM_STATE_LAST_ACK: - case NES_CM_STATE_CLOSING: - break; - case NES_CM_STATE_MPAREQ_SENT: - /* recv the mpa res frame, ret=frame len (incl priv data) */ - ret = parse_mpa(cm_node, dataloc, datasize); - if (ret < 0) - break; - /* set the req frame payload len in skb */ - /* we are done handling this state, set node to a TSA state */ - cm_node->state = NES_CM_STATE_TSA; - send_ack(cm_node); - create_event(cm_node, NES_CM_EVENT_CONNECTED); - break; - - case NES_CM_STATE_ESTABLISHED: - /* we are expecting an MPA req frame */ - ret = parse_mpa(cm_node, dataloc, datasize); - if (ret < 0) { - break; - } - cm_node->state = NES_CM_STATE_TSA; - send_ack(cm_node); - /* we got a valid MPA request, create an event */ - create_event(cm_node, NES_CM_EVENT_MPA_REQ); - break; - case NES_CM_STATE_TSA: - handle_exception_pkt(cm_node, skb); - break; - case NES_CM_STATE_UNKNOWN: - case NES_CM_STATE_INITED: - default: - ret = -1; - } - } +static void cleanup_retrans_entry(struct nes_cm_node *cm_node) +{ + unsigned long flags; - return ret; + spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + free_retrans_entry(cm_node); + spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); } +/** + * process_packet + * Returns skb if to be freed, else it will return NULL if already used.. + */ +static void process_packet(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct nes_cm_core *cm_core) +{ + enum nes_tcpip_pkt_type pkt_type = NES_PKT_TYPE_UNKNOWN; + struct tcphdr *tcph = tcp_hdr(skb); + skb_pull(skb, ip_hdr(skb)->ihl << 2); + + nes_debug(NES_DBG_CM, "process_packet: cm_node=%p state =%d syn=%d " + "ack=%d rst=%d fin=%d\n", cm_node, cm_node->state, tcph->syn, + tcph->ack, tcph->rst, tcph->fin); + + if (tcph->rst) + pkt_type = NES_PKT_TYPE_RST; + else if (tcph->syn) { + pkt_type = NES_PKT_TYPE_SYN; + if (tcph->ack) + pkt_type = NES_PKT_TYPE_SYNACK; + } else if (tcph->fin) + pkt_type = NES_PKT_TYPE_FIN; + else if (tcph->ack) + pkt_type = NES_PKT_TYPE_ACK; + + switch (pkt_type) { + case NES_PKT_TYPE_SYN: + handle_syn_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_SYNACK: + handle_synack_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_ACK: + handle_ack_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_RST: + handle_rst_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_FIN: + handle_fin_pkt(cm_node, skb, tcph); + break; + default: + drop_packet(skb); + break; + } +} /** * mini_cm_listen - create a listen node with params */ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, - struct nes_vnic *nesvnic, struct nes_cm_info *cm_info) + struct nes_vnic *nesvnic, struct nes_cm_info *cm_info) { struct nes_cm_listener *listener; unsigned long flags; @@ -1644,37 +1826,36 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, /** * mini_cm_connect - make a connection node with params */ -static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, - struct nes_vnic *nesvnic, - struct ietf_mpa_frame *mpa_frame, - struct nes_cm_info *cm_info) +struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, + struct nes_vnic *nesvnic, u16 private_data_len, + void *private_data, struct nes_cm_info *cm_info) { int ret = 0; struct nes_cm_node *cm_node; struct nes_cm_listener *loopbackremotelistener; struct nes_cm_node *loopbackremotenode; struct nes_cm_info loopback_cm_info; - - u16 mpa_frame_size = sizeof(struct ietf_mpa_frame) + - ntohs(mpa_frame->priv_data_len); - - cm_info->loc_addr = htonl(cm_info->loc_addr); - cm_info->rem_addr = htonl(cm_info->rem_addr); - cm_info->loc_port = htons(cm_info->loc_port); - cm_info->rem_port = htons(cm_info->rem_port); + u16 mpa_frame_size = sizeof(struct ietf_mpa_frame) + private_data_len; + struct ietf_mpa_frame *mpa_frame = NULL; /* create a CM connection node */ cm_node = make_cm_node(cm_core, nesvnic, cm_info, NULL); if (!cm_node) return NULL; + mpa_frame = &cm_node->mpa_frame; + strcpy(mpa_frame->key, IEFT_MPA_KEY_REQ); + mpa_frame->flags = IETF_MPA_FLAGS_CRC; + mpa_frame->rev = IETF_MPA_VERSION; + mpa_frame->priv_data_len = htons(private_data_len); /* set our node side to client (active) side */ cm_node->tcp_cntxt.client = 1; cm_node->tcp_cntxt.rcv_wscale = NES_CM_DEFAULT_RCV_WND_SCALE; if (cm_info->loc_addr == cm_info->rem_addr) { - loopbackremotelistener = find_listener(cm_core, cm_node->rem_addr, - cm_node->rem_port, NES_CM_LISTENER_ACTIVE_STATE); + loopbackremotelistener = find_listener(cm_core, + ntohl(nesvnic->local_ipaddr), cm_node->rem_port, + NES_CM_LISTENER_ACTIVE_STATE); if (loopbackremotelistener == NULL) { create_event(cm_node, NES_CM_EVENT_ABORTED); } else { @@ -1683,26 +1864,35 @@ static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, loopback_cm_info.loc_port = cm_info->rem_port; loopback_cm_info.rem_port = cm_info->loc_port; loopback_cm_info.cm_id = loopbackremotelistener->cm_id; - loopbackremotenode = make_cm_node(cm_core, nesvnic, &loopback_cm_info, - loopbackremotelistener); + loopbackremotenode = make_cm_node(cm_core, nesvnic, + &loopback_cm_info, loopbackremotelistener); loopbackremotenode->loopbackpartner = cm_node; - loopbackremotenode->tcp_cntxt.rcv_wscale = NES_CM_DEFAULT_RCV_WND_SCALE; + loopbackremotenode->tcp_cntxt.rcv_wscale = + NES_CM_DEFAULT_RCV_WND_SCALE; cm_node->loopbackpartner = loopbackremotenode; - memcpy(loopbackremotenode->mpa_frame_buf, &mpa_frame->priv_data, - mpa_frame_size); - loopbackremotenode->mpa_frame_size = mpa_frame_size - - sizeof(struct ietf_mpa_frame); + memcpy(loopbackremotenode->mpa_frame_buf, private_data, + private_data_len); + loopbackremotenode->mpa_frame_size = private_data_len; - /* we are done handling this state, set node to a TSA state */ + /* we are done handling this state. */ + /* set node to a TSA state */ cm_node->state = NES_CM_STATE_TSA; - cm_node->tcp_cntxt.rcv_nxt = loopbackremotenode->tcp_cntxt.loc_seq_num; - loopbackremotenode->tcp_cntxt.rcv_nxt = cm_node->tcp_cntxt.loc_seq_num; - cm_node->tcp_cntxt.max_snd_wnd = loopbackremotenode->tcp_cntxt.rcv_wnd; - loopbackremotenode->tcp_cntxt.max_snd_wnd = cm_node->tcp_cntxt.rcv_wnd; - cm_node->tcp_cntxt.snd_wnd = loopbackremotenode->tcp_cntxt.rcv_wnd; - loopbackremotenode->tcp_cntxt.snd_wnd = cm_node->tcp_cntxt.rcv_wnd; - cm_node->tcp_cntxt.snd_wscale = loopbackremotenode->tcp_cntxt.rcv_wscale; - loopbackremotenode->tcp_cntxt.snd_wscale = cm_node->tcp_cntxt.rcv_wscale; + cm_node->tcp_cntxt.rcv_nxt = + loopbackremotenode->tcp_cntxt.loc_seq_num; + loopbackremotenode->tcp_cntxt.rcv_nxt = + cm_node->tcp_cntxt.loc_seq_num; + cm_node->tcp_cntxt.max_snd_wnd = + loopbackremotenode->tcp_cntxt.rcv_wnd; + loopbackremotenode->tcp_cntxt.max_snd_wnd = + cm_node->tcp_cntxt.rcv_wnd; + cm_node->tcp_cntxt.snd_wnd = + loopbackremotenode->tcp_cntxt.rcv_wnd; + loopbackremotenode->tcp_cntxt.snd_wnd = + cm_node->tcp_cntxt.rcv_wnd; + cm_node->tcp_cntxt.snd_wscale = + loopbackremotenode->tcp_cntxt.rcv_wscale; + loopbackremotenode->tcp_cntxt.snd_wscale = + cm_node->tcp_cntxt.rcv_wscale; create_event(loopbackremotenode, NES_CM_EVENT_MPA_REQ); } @@ -1712,16 +1902,29 @@ static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, /* set our node side to client (active) side */ cm_node->tcp_cntxt.client = 1; /* init our MPA frame ptr */ - memcpy(&cm_node->mpa_frame, mpa_frame, mpa_frame_size); + memcpy(mpa_frame->priv_data, private_data, private_data_len); + cm_node->mpa_frame_size = mpa_frame_size; /* send a syn and goto syn sent state */ cm_node->state = NES_CM_STATE_SYN_SENT; - ret = send_syn(cm_node, 0); + ret = send_syn(cm_node, 0, NULL); + + if (ret) { + /* error in sending the syn free up the cm_node struct */ + nes_debug(NES_DBG_CM, "Api - connect() FAILED: dest " + "addr=0x%08X, port=0x%04x, cm_node=%p, cm_id = %p.\n", + cm_node->rem_addr, cm_node->rem_port, cm_node, + cm_node->cm_id); + rem_ref_cm_node(cm_node->cm_core, cm_node); + cm_node = NULL; + } - nes_debug(NES_DBG_CM, "Api - connect(): dest addr=0x%08X, port=0x%04x," - " cm_node=%p, cm_id = %p.\n", - cm_node->rem_addr, cm_node->rem_port, cm_node, cm_node->cm_id); + if (cm_node) + nes_debug(NES_DBG_CM, "Api - connect(): dest addr=0x%08X," + "port=0x%04x, cm_node=%p, cm_id = %p.\n", + cm_node->rem_addr, cm_node->rem_port, cm_node, + cm_node->cm_id); return cm_node; } @@ -1731,8 +1934,8 @@ static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, * mini_cm_accept - accept a connection * This function is never called */ -static int mini_cm_accept(struct nes_cm_core *cm_core, struct ietf_mpa_frame *mpa_frame, - struct nes_cm_node *cm_node) +static int mini_cm_accept(struct nes_cm_core *cm_core, + struct ietf_mpa_frame *mpa_frame, struct nes_cm_node *cm_node) { return 0; } @@ -1742,32 +1945,26 @@ static int mini_cm_accept(struct nes_cm_core *cm_core, struct ietf_mpa_frame *mp * mini_cm_reject - reject and teardown a connection */ static int mini_cm_reject(struct nes_cm_core *cm_core, - struct ietf_mpa_frame *mpa_frame, - struct nes_cm_node *cm_node) + struct ietf_mpa_frame *mpa_frame, struct nes_cm_node *cm_node) { int ret = 0; - struct sk_buff *skb; - u16 mpa_frame_size = sizeof(struct ietf_mpa_frame) + - ntohs(mpa_frame->priv_data_len); - skb = get_free_pkt(cm_node); - if (!skb) { - nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); - return -1; - } - - /* send an MPA Request frame */ - form_cm_frame(skb, cm_node, NULL, 0, mpa_frame, mpa_frame_size, SET_ACK | SET_FIN); - ret = schedule_nes_timer(cm_node, skb, NES_TIMER_TYPE_SEND, 1, 0); + nes_debug(NES_DBG_CM, "%s cm_node=%p type=%d state=%d\n", + __func__, cm_node, cm_node->tcp_cntxt.client, cm_node->state); + if (cm_node->tcp_cntxt.client) + return ret; + cleanup_retrans_entry(cm_node); cm_node->state = NES_CM_STATE_CLOSED; ret = send_fin(cm_node, NULL); - if (ret < 0) { - printk(KERN_INFO PFX "failed to send MPA Reply (reject)\n"); - return ret; + if (cm_node->accept_pend) { + BUG_ON(!cm_node->listener); + atomic_dec(&cm_node->listener->pend_accepts_cnt); + BUG_ON(atomic_read(&cm_node->listener->pend_accepts_cnt) < 0); } + ret = send_reset(cm_node, NULL); return ret; } @@ -1783,35 +1980,39 @@ static int mini_cm_close(struct nes_cm_core *cm_core, struct nes_cm_node *cm_nod return -EINVAL; switch (cm_node->state) { - /* if passed in node is null, create a reference key node for node search */ - /* check if we found an owner node for this pkt */ - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_ACCEPTING: - case NES_CM_STATE_MPAREQ_SENT: - cm_node->state = NES_CM_STATE_FIN_WAIT1; - send_fin(cm_node, NULL); - break; - case NES_CM_STATE_CLOSE_WAIT: - cm_node->state = NES_CM_STATE_LAST_ACK; - send_fin(cm_node, NULL); - break; - case NES_CM_STATE_FIN_WAIT1: - case NES_CM_STATE_FIN_WAIT2: - case NES_CM_STATE_LAST_ACK: - case NES_CM_STATE_TIME_WAIT: - case NES_CM_STATE_CLOSING: - ret = -1; - break; - case NES_CM_STATE_LISTENING: - case NES_CM_STATE_UNKNOWN: - case NES_CM_STATE_INITED: - case NES_CM_STATE_CLOSED: - case NES_CM_STATE_TSA: - ret = rem_ref_cm_node(cm_core, cm_node); - break; + case NES_CM_STATE_SYN_RCVD: + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_ONE_SIDE_ESTABLISHED: + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_ACCEPTING: + case NES_CM_STATE_MPAREQ_SENT: + case NES_CM_STATE_MPAREQ_RCVD: + cleanup_retrans_entry(cm_node); + send_reset(cm_node, NULL); + break; + case NES_CM_STATE_CLOSE_WAIT: + cm_node->state = NES_CM_STATE_LAST_ACK; + send_fin(cm_node, NULL); + break; + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_TIME_WAIT: + case NES_CM_STATE_CLOSING: + ret = -1; + break; + case NES_CM_STATE_LISTENING: + case NES_CM_STATE_UNKNOWN: + case NES_CM_STATE_INITED: + case NES_CM_STATE_CLOSED: + ret = rem_ref_cm_node(cm_core, cm_node); + break; + case NES_CM_STATE_TSA: + if (cm_node->send_entry) + printk(KERN_ERR "ERROR Close got called from STATE_TSA " + "send_entry=%p\n", cm_node->send_entry); + ret = rem_ref_cm_node(cm_core, cm_node); + break; } cm_node->cm_id = NULL; return ret; @@ -1822,25 +2023,30 @@ static int mini_cm_close(struct nes_cm_core *cm_core, struct nes_cm_node *cm_nod * recv_pkt - recv an ETHERNET packet, and process it through CM * node state machine */ -static int mini_cm_recv_pkt(struct nes_cm_core *cm_core, struct nes_vnic *nesvnic, - struct sk_buff *skb) +static void mini_cm_recv_pkt(struct nes_cm_core *cm_core, + struct nes_vnic *nesvnic, struct sk_buff *skb) { struct nes_cm_node *cm_node = NULL; struct nes_cm_listener *listener = NULL; struct iphdr *iph; struct tcphdr *tcph; struct nes_cm_info nfo; - int ret = 0; - if (!skb || skb->len < sizeof(struct iphdr) + sizeof(struct tcphdr)) { - ret = -EINVAL; - goto out; + if (!skb) + return; + if (skb->len < sizeof(struct iphdr) + sizeof(struct tcphdr)) { + dev_kfree_skb_any(skb); + return; } iph = (struct iphdr *)skb->data; tcph = (struct tcphdr *)(skb->data + sizeof(struct iphdr)); skb_reset_network_header(skb); skb_set_transport_header(skb, sizeof(*tcph)); + if (!tcph) { + dev_kfree_skb_any(skb); + return; + } skb->len = ntohs(iph->tot_len); nfo.loc_addr = ntohl(iph->daddr); @@ -1853,61 +2059,60 @@ static int mini_cm_recv_pkt(struct nes_cm_core *cm_core, struct nes_vnic *nesvni NIPQUAD(iph->daddr), tcph->dest, NIPQUAD(iph->saddr), tcph->source); - /* note: this call is going to increment cm_node ref count */ - cm_node = find_node(cm_core, + do { + cm_node = find_node(cm_core, nfo.rem_port, nfo.rem_addr, nfo.loc_port, nfo.loc_addr); - if (!cm_node) { - listener = find_listener(cm_core, nfo.loc_addr, nfo.loc_port, - NES_CM_LISTENER_ACTIVE_STATE); - if (listener) { - nfo.cm_id = listener->cm_id; - nfo.conn_type = listener->conn_type; - } else { - nfo.cm_id = NULL; - nfo.conn_type = 0; - } - - cm_node = make_cm_node(cm_core, nesvnic, &nfo, listener); if (!cm_node) { - nes_debug(NES_DBG_CM, "Unable to allocate node\n"); + /* Only type of packet accepted are for */ + /* the PASSIVE open (syn only) */ + if ((!tcph->syn) || (tcph->ack)) { + cm_packets_dropped++; + break; + } + listener = find_listener(cm_core, nfo.loc_addr, + nfo.loc_port, + NES_CM_LISTENER_ACTIVE_STATE); if (listener) { - nes_debug(NES_DBG_CM, "unable to allocate node and decrementing listener refcount\n"); + nfo.cm_id = listener->cm_id; + nfo.conn_type = listener->conn_type; + } else { + nes_debug(NES_DBG_CM, "Unable to find listener " + "for the pkt\n"); + cm_packets_dropped++; + dev_kfree_skb_any(skb); + break; + } + + cm_node = make_cm_node(cm_core, nesvnic, &nfo, + listener); + if (!cm_node) { + nes_debug(NES_DBG_CM, "Unable to allocate " + "node\n"); + cm_packets_dropped++; atomic_dec(&listener->ref_count); + dev_kfree_skb_any(skb); + break; } - ret = -1; - goto out; - } - if (!listener) { - nes_debug(NES_DBG_CM, "Packet found for unknown port %x refcnt=%d\n", - nfo.loc_port, atomic_read(&cm_node->ref_count)); - if (!tcph->rst) { - nes_debug(NES_DBG_CM, "Packet found for unknown port=%d" - " rem_port=%d refcnt=%d\n", - nfo.loc_port, nfo.rem_port, atomic_read(&cm_node->ref_count)); - - cm_node->tcp_cntxt.rcv_nxt = ntohl(tcph->seq); - cm_node->tcp_cntxt.loc_seq_num = ntohl(tcph->ack_seq); - send_reset(cm_node); + if (!tcph->rst && !tcph->fin) { + cm_node->state = NES_CM_STATE_LISTENING; + } else { + cm_packets_dropped++; + rem_ref_cm_node(cm_core, cm_node); + dev_kfree_skb_any(skb); + break; } + add_ref_cm_node(cm_node); + } else if (cm_node->state == NES_CM_STATE_TSA) { rem_ref_cm_node(cm_core, cm_node); - ret = -1; - goto out; + atomic_inc(&cm_accel_dropped_pkts); + dev_kfree_skb_any(skb); + break; } - add_ref_cm_node(cm_node); - cm_node->state = NES_CM_STATE_LISTENING; - } - - nes_debug(NES_DBG_CM, "Processing Packet for node %p, data = (%p):\n", - cm_node, skb->data); - process_packet(cm_node, skb, cm_core); - - rem_ref_cm_node(cm_core, cm_node); - out: - if (skb) - dev_kfree_skb_any(skb); - return ret; + process_packet(cm_node, skb, cm_core); + rem_ref_cm_node(cm_core, cm_node); + } while (0); } @@ -2107,15 +2312,12 @@ int nes_cm_disconn(struct nes_qp *nesqp) if (nesqp->disconn_pending == 0) { nesqp->disconn_pending++; spin_unlock_irqrestore(&nesqp->lock, flags); - /* nes_add_ref(&nesqp->ibqp); */ /* init our disconnect work element, to */ INIT_WORK(&nesqp->disconn_work, nes_disconnect_worker); queue_work(g_cm_core->disconn_wq, &nesqp->disconn_work); - } else { + } else spin_unlock_irqrestore(&nesqp->lock, flags); - nes_rem_ref(&nesqp->ibqp); - } return 0; } @@ -2161,7 +2363,6 @@ static int nes_cm_disconn_true(struct nes_qp *nesqp) nes_debug(NES_DBG_CM, "QP%u disconnect_worker cmid is NULL\n", nesqp->hwqp.qp_id); spin_unlock_irqrestore(&nesqp->lock, flags); - nes_rem_ref(&nesqp->ibqp); return -1; } @@ -2182,30 +2383,31 @@ static int nes_cm_disconn_true(struct nes_qp *nesqp) atomic_inc(&cm_disconnects); cm_event.event = IW_CM_EVENT_DISCONNECT; if (last_ae == NES_AEQE_AEID_LLP_CONNECTION_RESET) { - issued_disconnect_reset = 1; cm_event.status = IW_CM_EVENT_STATUS_RESET; - nes_debug(NES_DBG_CM, "Generating a CM Disconnect Event (status reset) for " - " QP%u, cm_id = %p. \n", - nesqp->hwqp.qp_id, cm_id); - } else { + nes_debug(NES_DBG_CM, "Generating a CM " + "Disconnect Event (status reset) for " + "QP%u, cm_id = %p. \n", + nesqp->hwqp.qp_id, cm_id); + } else cm_event.status = IW_CM_EVENT_STATUS_OK; - } cm_event.local_addr = cm_id->local_addr; cm_event.remote_addr = cm_id->remote_addr; cm_event.private_data = NULL; cm_event.private_data_len = 0; - nes_debug(NES_DBG_CM, "Generating a CM Disconnect Event for " - " QP%u, SQ Head = %u, SQ Tail = %u. cm_id = %p, refcount = %u.\n", - nesqp->hwqp.qp_id, - nesqp->hwqp.sq_head, nesqp->hwqp.sq_tail, cm_id, - atomic_read(&nesqp->refcount)); + nes_debug(NES_DBG_CM, "Generating a CM Disconnect Event" + " for QP%u, SQ Head = %u, SQ Tail = %u. " + "cm_id = %p, refcount = %u.\n", + nesqp->hwqp.qp_id, nesqp->hwqp.sq_head, + nesqp->hwqp.sq_tail, cm_id, + atomic_read(&nesqp->refcount)); spin_unlock_irqrestore(&nesqp->lock, flags); ret = cm_id->event_handler(cm_id, &cm_event); if (ret) - nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret); + nes_debug(NES_DBG_CM, "OFA CM event_handler " + "returned, ret=%d\n", ret); spin_lock_irqsave(&nesqp->lock, flags); } @@ -2247,31 +2449,24 @@ static int nes_cm_disconn_true(struct nes_qp *nesqp) if (nesqp->flush_issued == 0) { nesqp->flush_issued = 1; spin_unlock_irqrestore(&nesqp->lock, flags); - flush_wqes(nesvnic->nesdev, nesqp, NES_CQP_FLUSH_RQ, 1); - } else { + flush_wqes(nesvnic->nesdev, nesqp, + NES_CQP_FLUSH_RQ, 1); + } else spin_unlock_irqrestore(&nesqp->lock, flags); - } - - /* This reference is from either ModifyQP or the AE processing, - there is still a race here with modifyqp */ - nes_rem_ref(&nesqp->ibqp); - } else { cm_id = nesqp->cm_id; spin_unlock_irqrestore(&nesqp->lock, flags); /* check to see if the inbound reset beat the outbound reset */ if ((!cm_id) && (last_ae==NES_AEQE_AEID_RESET_SENT)) { - nes_debug(NES_DBG_CM, "QP%u: Decing refcount due to inbound reset" - " beating the outbound reset.\n", - nesqp->hwqp.qp_id); - nes_rem_ref(&nesqp->ibqp); + nes_debug(NES_DBG_CM, "QP%u: Decing refcount " + "due to inbound reset beating the " + "outbound reset.\n", nesqp->hwqp.qp_id); } } } else { nesqp->disconn_pending = 0; spin_unlock_irqrestore(&nesqp->lock, flags); } - nes_rem_ref(&nesqp->ibqp); return 0; } @@ -2349,71 +2544,82 @@ int nes_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) nesdev = nesvnic->nesdev; adapter = nesdev->nesadapter; - nes_debug(NES_DBG_CM, "nesvnic=%p, netdev=%p, %s\n", - nesvnic, nesvnic->netdev, nesvnic->netdev->name); - - /* since this is from a listen, we were able to put node handle into cm_id */ cm_node = (struct nes_cm_node *)cm_id->provider_data; + nes_debug(NES_DBG_CM, "nes_accept: cm_node= %p nesvnic=%p, netdev=%p," + "%s\n", cm_node, nesvnic, nesvnic->netdev, + nesvnic->netdev->name); /* associate the node with the QP */ nesqp->cm_node = (void *)cm_node; + cm_node->nesqp = nesqp; + nes_add_ref(&nesqp->ibqp); - nes_debug(NES_DBG_CM, "QP%u, cm_node=%p, jiffies = %lu\n", - nesqp->hwqp.qp_id, cm_node, jiffies); + nes_debug(NES_DBG_CM, "QP%u, cm_node=%p, jiffies = %lu listener = %p\n", + nesqp->hwqp.qp_id, cm_node, jiffies, cm_node->listener); atomic_inc(&cm_accepts); nes_debug(NES_DBG_CM, "netdev refcnt = %u.\n", atomic_read(&nesvnic->netdev->refcnt)); - /* allocate the ietf frame and space for private data */ - nesqp->ietf_frame = pci_alloc_consistent(nesdev->pcidev, - sizeof(struct ietf_mpa_frame) + conn_param->private_data_len, - &nesqp->ietf_frame_pbase); - - if (!nesqp->ietf_frame) { - nes_debug(NES_DBG_CM, "Unable to allocate memory for private data\n"); - return -ENOMEM; - } + /* allocate the ietf frame and space for private data */ + nesqp->ietf_frame = pci_alloc_consistent(nesdev->pcidev, + sizeof(struct ietf_mpa_frame) + conn_param->private_data_len, + &nesqp->ietf_frame_pbase); + if (!nesqp->ietf_frame) { + nes_debug(NES_DBG_CM, "Unable to allocate memory for private " + "data\n"); + return -ENOMEM; + } - /* setup the MPA frame */ - nesqp->private_data_len = conn_param->private_data_len; - memcpy(nesqp->ietf_frame->key, IEFT_MPA_KEY_REP, IETF_MPA_KEY_SIZE); - memcpy(nesqp->ietf_frame->priv_data, conn_param->private_data, - conn_param->private_data_len); + /* setup the MPA frame */ + nesqp->private_data_len = conn_param->private_data_len; + memcpy(nesqp->ietf_frame->key, IEFT_MPA_KEY_REP, IETF_MPA_KEY_SIZE); - nesqp->ietf_frame->priv_data_len = cpu_to_be16(conn_param->private_data_len); - nesqp->ietf_frame->rev = mpa_version; - nesqp->ietf_frame->flags = IETF_MPA_FLAGS_CRC; + memcpy(nesqp->ietf_frame->priv_data, conn_param->private_data, + conn_param->private_data_len); - /* setup our first outgoing iWarp send WQE (the IETF frame response) */ - wqe = &nesqp->hwqp.sq_vbase[0]; + nesqp->ietf_frame->priv_data_len = + cpu_to_be16(conn_param->private_data_len); + nesqp->ietf_frame->rev = mpa_version; + nesqp->ietf_frame->flags = IETF_MPA_FLAGS_CRC; - if (cm_id->remote_addr.sin_addr.s_addr != cm_id->local_addr.sin_addr.s_addr) { - u64temp = (unsigned long)nesqp; - u64temp |= NES_SW_CONTEXT_ALIGN>>1; - set_wqe_64bit_value(wqe->wqe_words, NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, - u64temp); - wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = - cpu_to_le32(NES_IWARP_SQ_WQE_STREAMING | NES_IWARP_SQ_WQE_WRPDU); - wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = - cpu_to_le32(conn_param->private_data_len + sizeof(struct ietf_mpa_frame)); - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = - cpu_to_le32((u32)nesqp->ietf_frame_pbase); - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = - cpu_to_le32((u32)((u64)nesqp->ietf_frame_pbase >> 32)); - wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = - cpu_to_le32(conn_param->private_data_len + sizeof(struct ietf_mpa_frame)); - wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; - - nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32( - NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | NES_QPCONTEXT_ORDIRD_WRPDU); - } else { - nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32((NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | - NES_QPCONTEXT_ORDIRD_WRPDU | NES_QPCONTEXT_ORDIRD_ALSMM)); - } - nesqp->skip_lsmm = 1; + /* setup our first outgoing iWarp send WQE (the IETF frame response) */ + wqe = &nesqp->hwqp.sq_vbase[0]; + + if (cm_id->remote_addr.sin_addr.s_addr != + cm_id->local_addr.sin_addr.s_addr) { + u64temp = (unsigned long)nesqp; + u64temp |= NES_SW_CONTEXT_ALIGN>>1; + set_wqe_64bit_value(wqe->wqe_words, + NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, + u64temp); + wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = + cpu_to_le32(NES_IWARP_SQ_WQE_STREAMING | + NES_IWARP_SQ_WQE_WRPDU); + wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = + cpu_to_le32(conn_param->private_data_len + + sizeof(struct ietf_mpa_frame)); + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = + cpu_to_le32((u32)nesqp->ietf_frame_pbase); + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = + cpu_to_le32((u32)((u64)nesqp->ietf_frame_pbase >> 32)); + wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = + cpu_to_le32(conn_param->private_data_len + + sizeof(struct ietf_mpa_frame)); + wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; + + nesqp->nesqp_context->ird_ord_sizes |= + cpu_to_le32(NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | + NES_QPCONTEXT_ORDIRD_WRPDU); + } else { + nesqp->nesqp_context->ird_ord_sizes |= + cpu_to_le32((NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | + NES_QPCONTEXT_ORDIRD_WRPDU | + NES_QPCONTEXT_ORDIRD_ALSMM)); + } + nesqp->skip_lsmm = 1; /* Cache the cm_id in the qp */ @@ -2424,55 +2630,75 @@ int nes_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) cm_id->provider_data = nesqp; nesqp->active_conn = 0; + if (cm_node->state == NES_CM_STATE_TSA) + nes_debug(NES_DBG_CM, "Already state = TSA for cm_node=%p\n", + cm_node); + nes_cm_init_tsa_conn(nesqp, cm_node); - nesqp->nesqp_context->tcpPorts[0] = cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); - nesqp->nesqp_context->tcpPorts[1] = cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); - nesqp->nesqp_context->ip0 = cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); + nesqp->nesqp_context->tcpPorts[0] = + cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); + nesqp->nesqp_context->tcpPorts[1] = + cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); + + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(nesvnic->local_ipaddr)); + else + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); nesqp->nesqp_context->misc2 |= cpu_to_le32( - (u32)PCI_FUNC(nesdev->pcidev->devfn) << NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); + (u32)PCI_FUNC(nesdev->pcidev->devfn) << + NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); - nesqp->nesqp_context->arp_index_vlan |= cpu_to_le32( - nes_arp_table(nesdev, le32_to_cpu(nesqp->nesqp_context->ip0), NULL, + nesqp->nesqp_context->arp_index_vlan |= + cpu_to_le32(nes_arp_table(nesdev, + le32_to_cpu(nesqp->nesqp_context->ip0), NULL, NES_ARP_RESOLVE) << 16); nesqp->nesqp_context->ts_val_delta = cpu_to_le32( - jiffies - nes_read_indexed(nesdev, NES_IDX_TCP_NOW)); + jiffies - nes_read_indexed(nesdev, NES_IDX_TCP_NOW)); nesqp->nesqp_context->ird_index = cpu_to_le32(nesqp->hwqp.qp_id); nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32( - ((u32)1 << NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT)); - nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32((u32)conn_param->ord); + ((u32)1 << NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT)); + nesqp->nesqp_context->ird_ord_sizes |= + cpu_to_le32((u32)conn_param->ord); memset(&nes_quad, 0, sizeof(nes_quad)); - nes_quad.DstIpAdrIndex = cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); - nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; - nes_quad.TcpPorts[0] = cm_id->remote_addr.sin_port; - nes_quad.TcpPorts[1] = cm_id->local_addr.sin_port; + nes_quad.DstIpAdrIndex = + cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nes_quad.SrcIpadr = nesvnic->local_ipaddr; + else + nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; + nes_quad.TcpPorts[0] = cm_id->remote_addr.sin_port; + nes_quad.TcpPorts[1] = cm_id->local_addr.sin_port; /* Produce hash key */ crc_value = get_crc_value(&nes_quad); nesqp->hte_index = cpu_to_be32(crc_value ^ 0xffffffff); nes_debug(NES_DBG_CM, "HTE Index = 0x%08X, CRC = 0x%08X\n", - nesqp->hte_index, nesqp->hte_index & adapter->hte_index_mask); + nesqp->hte_index, nesqp->hte_index & adapter->hte_index_mask); nesqp->hte_index &= adapter->hte_index_mask; nesqp->nesqp_context->hte_index = cpu_to_le32(nesqp->hte_index); cm_node->cm_core->api->accelerated(cm_node->cm_core, cm_node); - nes_debug(NES_DBG_CM, "QP%u, Destination IP = 0x%08X:0x%04X, local = 0x%08X:0x%04X," - " rcv_nxt=0x%08X, snd_nxt=0x%08X, mpa + private data length=%zu.\n", - nesqp->hwqp.qp_id, + nes_debug(NES_DBG_CM, "QP%u, Destination IP = 0x%08X:0x%04X, local = " + "0x%08X:0x%04X, rcv_nxt=0x%08X, snd_nxt=0x%08X, mpa + " + "private data length=%zu.\n", nesqp->hwqp.qp_id, ntohl(cm_id->remote_addr.sin_addr.s_addr), ntohs(cm_id->remote_addr.sin_port), ntohl(cm_id->local_addr.sin_addr.s_addr), ntohs(cm_id->local_addr.sin_port), le32_to_cpu(nesqp->nesqp_context->rcv_nxt), le32_to_cpu(nesqp->nesqp_context->snd_nxt), - conn_param->private_data_len+sizeof(struct ietf_mpa_frame)); + conn_param->private_data_len + + sizeof(struct ietf_mpa_frame)); attr.qp_state = IB_QPS_RTS; nes_modify_qp(&nesqp->ibqp, &attr, IB_QP_STATE, NULL); @@ -2489,15 +2715,16 @@ int nes_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) cm_event.private_data_len = 0; ret = cm_id->event_handler(cm_id, &cm_event); if (cm_node->loopbackpartner) { - cm_node->loopbackpartner->mpa_frame_size = nesqp->private_data_len; + cm_node->loopbackpartner->mpa_frame_size = + nesqp->private_data_len; /* copy entire MPA frame to our cm_node's frame */ - memcpy(cm_node->loopbackpartner->mpa_frame_buf, nesqp->ietf_frame->priv_data, - nesqp->private_data_len); + memcpy(cm_node->loopbackpartner->mpa_frame_buf, + nesqp->ietf_frame->priv_data, nesqp->private_data_len); create_event(cm_node->loopbackpartner, NES_CM_EVENT_CONNECTED); } if (ret) - printk("%s[%u] OFA CM event_handler returned, ret=%d\n", - __func__, __LINE__, ret); + printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " + "ret=%d\n", __func__, __LINE__, ret); return 0; } @@ -2555,74 +2782,61 @@ int nes_connect(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) if (!nesdev) return -EINVAL; - atomic_inc(&cm_connects); - - nesqp->ietf_frame = kzalloc(sizeof(struct ietf_mpa_frame) + - conn_param->private_data_len, GFP_KERNEL); - if (!nesqp->ietf_frame) - return -ENOMEM; + nes_debug(NES_DBG_CM, "QP%u, current IP = 0x%08X, Destination IP = " + "0x%08X:0x%04X, local = 0x%08X:0x%04X.\n", nesqp->hwqp.qp_id, + ntohl(nesvnic->local_ipaddr), + ntohl(cm_id->remote_addr.sin_addr.s_addr), + ntohs(cm_id->remote_addr.sin_port), + ntohl(cm_id->local_addr.sin_addr.s_addr), + ntohs(cm_id->local_addr.sin_port)); - /* set qp as having an active connection */ + atomic_inc(&cm_connects); nesqp->active_conn = 1; - nes_debug(NES_DBG_CM, "QP%u, Destination IP = 0x%08X:0x%04X, local = 0x%08X:0x%04X.\n", - nesqp->hwqp.qp_id, - ntohl(cm_id->remote_addr.sin_addr.s_addr), - ntohs(cm_id->remote_addr.sin_port), - ntohl(cm_id->local_addr.sin_addr.s_addr), - ntohs(cm_id->local_addr.sin_port)); - /* cache the cm_id in the qp */ nesqp->cm_id = cm_id; cm_id->provider_data = nesqp; - /* copy the private data */ - if (conn_param->private_data_len) { - memcpy(nesqp->ietf_frame->priv_data, conn_param->private_data, - conn_param->private_data_len); - } - nesqp->private_data_len = conn_param->private_data_len; nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32((u32)conn_param->ord); nes_debug(NES_DBG_CM, "requested ord = 0x%08X.\n", (u32)conn_param->ord); - nes_debug(NES_DBG_CM, "mpa private data len =%u\n", conn_param->private_data_len); - - strcpy(&nesqp->ietf_frame->key[0], IEFT_MPA_KEY_REQ); - nesqp->ietf_frame->flags = IETF_MPA_FLAGS_CRC; - nesqp->ietf_frame->rev = IETF_MPA_VERSION; - nesqp->ietf_frame->priv_data_len = htons(conn_param->private_data_len); + nes_debug(NES_DBG_CM, "mpa private data len =%u\n", + conn_param->private_data_len); - if (cm_id->local_addr.sin_addr.s_addr != cm_id->remote_addr.sin_addr.s_addr) + if (cm_id->local_addr.sin_addr.s_addr != + cm_id->remote_addr.sin_addr.s_addr) nes_manage_apbvt(nesvnic, ntohs(cm_id->local_addr.sin_port), - PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD); + PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD); /* set up the connection params for the node */ - cm_info.loc_addr = (cm_id->local_addr.sin_addr.s_addr); - cm_info.loc_port = (cm_id->local_addr.sin_port); - cm_info.rem_addr = (cm_id->remote_addr.sin_addr.s_addr); - cm_info.rem_port = (cm_id->remote_addr.sin_port); + cm_info.loc_addr = htonl(cm_id->local_addr.sin_addr.s_addr); + cm_info.loc_port = htons(cm_id->local_addr.sin_port); + cm_info.rem_addr = htonl(cm_id->remote_addr.sin_addr.s_addr); + cm_info.rem_port = htons(cm_id->remote_addr.sin_port); cm_info.cm_id = cm_id; cm_info.conn_type = NES_CM_IWARP_CONN_TYPE; cm_id->add_ref(cm_id); - nes_add_ref(&nesqp->ibqp); /* create a connect CM node connection */ - cm_node = g_cm_core->api->connect(g_cm_core, nesvnic, nesqp->ietf_frame, &cm_info); + cm_node = g_cm_core->api->connect(g_cm_core, nesvnic, + conn_param->private_data_len, (void *)conn_param->private_data, + &cm_info); if (!cm_node) { - if (cm_id->local_addr.sin_addr.s_addr != cm_id->remote_addr.sin_addr.s_addr) + if (cm_id->local_addr.sin_addr.s_addr != + cm_id->remote_addr.sin_addr.s_addr) nes_manage_apbvt(nesvnic, ntohs(cm_id->local_addr.sin_port), - PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_DEL); - nes_rem_ref(&nesqp->ibqp); - kfree(nesqp->ietf_frame); - nesqp->ietf_frame = NULL; + PCI_FUNC(nesdev->pcidev->devfn), + NES_MANAGE_APBVT_DEL); + cm_id->rem_ref(cm_id); return -ENOMEM; } cm_node->apbvt_set = 1; nesqp->cm_node = cm_node; + cm_node->nesqp = nesqp; return 0; } @@ -2664,7 +2878,7 @@ int nes_create_listen(struct iw_cm_id *cm_id, int backlog) cm_node = g_cm_core->api->listen(g_cm_core, nesvnic, &cm_info); if (!cm_node) { - printk("%s[%u] Error returned from listen API call\n", + printk(KERN_ERR "%s[%u] Error returned from listen API call\n", __func__, __LINE__); return -ENOMEM; } @@ -2672,10 +2886,13 @@ int nes_create_listen(struct iw_cm_id *cm_id, int backlog) cm_id->provider_data = cm_node; if (!cm_node->reused_node) { - err = nes_manage_apbvt(nesvnic, ntohs(cm_id->local_addr.sin_port), - PCI_FUNC(nesvnic->nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD); + err = nes_manage_apbvt(nesvnic, + ntohs(cm_id->local_addr.sin_port), + PCI_FUNC(nesvnic->nesdev->pcidev->devfn), + NES_MANAGE_APBVT_ADD); if (err) { - printk("nes_manage_apbvt call returned %d.\n", err); + printk(KERN_ERR "nes_manage_apbvt call returned %d.\n", + err); g_cm_core->api->stop_listener(g_cm_core, (void *)cm_node); return err; } @@ -2795,53 +3012,70 @@ static void cm_event_connected(struct nes_cm_event *event) nes_cm_init_tsa_conn(nesqp, cm_node); /* set the QP tsa context */ - nesqp->nesqp_context->tcpPorts[0] = cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); - nesqp->nesqp_context->tcpPorts[1] = cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); - nesqp->nesqp_context->ip0 = cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); + nesqp->nesqp_context->tcpPorts[0] = + cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); + nesqp->nesqp_context->tcpPorts[1] = + cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(nesvnic->local_ipaddr)); + else + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); nesqp->nesqp_context->misc2 |= cpu_to_le32( - (u32)PCI_FUNC(nesdev->pcidev->devfn) << NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); + (u32)PCI_FUNC(nesdev->pcidev->devfn) << + NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); nesqp->nesqp_context->arp_index_vlan |= cpu_to_le32( - nes_arp_table(nesdev, le32_to_cpu(nesqp->nesqp_context->ip0), + nes_arp_table(nesdev, + le32_to_cpu(nesqp->nesqp_context->ip0), NULL, NES_ARP_RESOLVE) << 16); nesqp->nesqp_context->ts_val_delta = cpu_to_le32( jiffies - nes_read_indexed(nesdev, NES_IDX_TCP_NOW)); nesqp->nesqp_context->ird_index = cpu_to_le32(nesqp->hwqp.qp_id); nesqp->nesqp_context->ird_ord_sizes |= - cpu_to_le32((u32)1 << NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT); + cpu_to_le32((u32)1 << + NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT); /* Adjust tail for not having a LSMM */ nesqp->hwqp.sq_tail = 1; #if defined(NES_SEND_FIRST_WRITE) - if (cm_node->send_write0) { - nes_debug(NES_DBG_CM, "Sending first write.\n"); - wqe = &nesqp->hwqp.sq_vbase[0]; - u64temp = (unsigned long)nesqp; - u64temp |= NES_SW_CONTEXT_ALIGN>>1; - set_wqe_64bit_value(wqe->wqe_words, NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, - u64temp); - wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = cpu_to_le32(NES_IWARP_SQ_OP_RDMAW); - wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; - - /* use the reserved spot on the WQ for the extra first WQE */ - nesqp->nesqp_context->ird_ord_sizes &= cpu_to_le32(~(NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | - NES_QPCONTEXT_ORDIRD_WRPDU | NES_QPCONTEXT_ORDIRD_ALSMM)); - nesqp->skip_lsmm = 1; - nesqp->hwqp.sq_tail = 0; - nes_write32(nesdev->regs + NES_WQE_ALLOC, - (1 << 24) | 0x00800000 | nesqp->hwqp.qp_id); - } + if (cm_node->send_write0) { + nes_debug(NES_DBG_CM, "Sending first write.\n"); + wqe = &nesqp->hwqp.sq_vbase[0]; + u64temp = (unsigned long)nesqp; + u64temp |= NES_SW_CONTEXT_ALIGN>>1; + set_wqe_64bit_value(wqe->wqe_words, + NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, u64temp); + wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = + cpu_to_le32(NES_IWARP_SQ_OP_RDMAW); + wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; + + /* use the reserved spot on the WQ for the extra first WQE */ + nesqp->nesqp_context->ird_ord_sizes &= + cpu_to_le32(~(NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | + NES_QPCONTEXT_ORDIRD_WRPDU | + NES_QPCONTEXT_ORDIRD_ALSMM)); + nesqp->skip_lsmm = 1; + nesqp->hwqp.sq_tail = 0; + nes_write32(nesdev->regs + NES_WQE_ALLOC, + (1 << 24) | 0x00800000 | nesqp->hwqp.qp_id); + } #endif memset(&nes_quad, 0, sizeof(nes_quad)); - nes_quad.DstIpAdrIndex = cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); - nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; + nes_quad.DstIpAdrIndex = + cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nes_quad.SrcIpadr = nesvnic->local_ipaddr; + else + nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; nes_quad.TcpPorts[0] = cm_id->remote_addr.sin_port; nes_quad.TcpPorts[1] = cm_id->local_addr.sin_port; @@ -2858,10 +3092,6 @@ static void cm_event_connected(struct nes_cm_event *event) nesqp->private_data_len = (u8) cm_node->mpa_frame_size; cm_node->cm_core->api->accelerated(cm_node->cm_core, cm_node); - /* modify QP state to rts */ - attr.qp_state = IB_QPS_RTS; - nes_modify_qp(&nesqp->ibqp, &attr, IB_QP_STATE, NULL); - /* notify OF layer we successfully created the requested connection */ cm_event.event = IW_CM_EVENT_CONNECT_REPLY; cm_event.status = IW_CM_EVENT_STATUS_ACCEPTED; @@ -2870,20 +3100,21 @@ static void cm_event_connected(struct nes_cm_event *event) cm_event.local_addr.sin_port = cm_id->local_addr.sin_port; cm_event.remote_addr = cm_id->remote_addr; - cm_event.private_data = (void *)event->cm_node->mpa_frame_buf; - cm_event.private_data_len = (u8) event->cm_node->mpa_frame_size; + cm_event.private_data = (void *)event->cm_node->mpa_frame_buf; + cm_event.private_data_len = (u8) event->cm_node->mpa_frame_size; cm_event.local_addr.sin_addr.s_addr = event->cm_info.rem_addr; ret = cm_id->event_handler(cm_id, &cm_event); nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret); if (ret) - printk("%s[%u] OFA CM event_handler returned, ret=%d\n", - __func__, __LINE__, ret); - nes_debug(NES_DBG_CM, "Exiting connect thread for QP%u. jiffies = %lu\n", - nesqp->hwqp.qp_id, jiffies ); + printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " + "ret=%d\n", __func__, __LINE__, ret); + attr.qp_state = IB_QPS_RTS; + nes_modify_qp(&nesqp->ibqp, &attr, IB_QP_STATE, NULL); - nes_rem_ref(&nesqp->ibqp); + nes_debug(NES_DBG_CM, "Exiting connect thread for QP%u. jiffies = " + "%lu\n", nesqp->hwqp.qp_id, jiffies); return; } @@ -2927,17 +3158,19 @@ static void cm_event_connect_error(struct nes_cm_event *event) cm_event.private_data = NULL; cm_event.private_data_len = 0; - nes_debug(NES_DBG_CM, "call CM_EVENT REJECTED, local_addr=%08x, remove_addr=%08x\n", - cm_event.local_addr.sin_addr.s_addr, cm_event.remote_addr.sin_addr.s_addr); + nes_debug(NES_DBG_CM, "call CM_EVENT REJECTED, local_addr=%08x, " + "remove_addr=%08x\n", cm_event.local_addr.sin_addr.s_addr, + cm_event.remote_addr.sin_addr.s_addr); ret = cm_id->event_handler(cm_id, &cm_event); nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret); if (ret) - printk("%s[%u] OFA CM event_handler returned, ret=%d\n", - __func__, __LINE__, ret); + printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " + "ret=%d\n", __func__, __LINE__, ret); nes_rem_ref(&nesqp->ibqp); - cm_id->rem_ref(cm_id); + cm_id->rem_ref(cm_id); + rem_ref_cm_node(event->cm_node->cm_core, event->cm_node); return; } @@ -3040,7 +3273,8 @@ static int nes_cm_post_event(struct nes_cm_event *event) add_ref_cm_node(event->cm_node); event->cm_info.cm_id->add_ref(event->cm_info.cm_id); INIT_WORK(&event->event_work, nes_cm_event_handler); - nes_debug(NES_DBG_CM, "queue_work, event=%p\n", event); + nes_debug(NES_DBG_CM, "cm_node=%p queue_work, event=%p\n", + event->cm_node, event); queue_work(event->cm_node->cm_core->event_wq, &event->event_work); @@ -3056,46 +3290,48 @@ static int nes_cm_post_event(struct nes_cm_event *event) */ static void nes_cm_event_handler(struct work_struct *work) { - struct nes_cm_event *event = container_of(work, struct nes_cm_event, event_work); + struct nes_cm_event *event = container_of(work, struct nes_cm_event, + event_work); struct nes_cm_core *cm_core; - if ((!event) || (!event->cm_node) || (!event->cm_node->cm_core)) { + if ((!event) || (!event->cm_node) || (!event->cm_node->cm_core)) return; - } + cm_core = event->cm_node->cm_core; nes_debug(NES_DBG_CM, "event=%p, event->type=%u, events posted=%u\n", - event, event->type, atomic_read(&cm_core->events_posted)); + event, event->type, atomic_read(&cm_core->events_posted)); switch (event->type) { - case NES_CM_EVENT_MPA_REQ: - cm_event_mpa_req(event); - nes_debug(NES_DBG_CM, "CM Event: MPA REQUEST\n"); - break; - case NES_CM_EVENT_RESET: - nes_debug(NES_DBG_CM, "CM Event: RESET\n"); - cm_event_reset(event); - break; - case NES_CM_EVENT_CONNECTED: - if ((!event->cm_node->cm_id) || - (event->cm_node->state != NES_CM_STATE_TSA)) { - break; - } - cm_event_connected(event); - nes_debug(NES_DBG_CM, "CM Event: CONNECTED\n"); + case NES_CM_EVENT_MPA_REQ: + cm_event_mpa_req(event); + nes_debug(NES_DBG_CM, "cm_node=%p CM Event: MPA REQUEST\n", + event->cm_node); + break; + case NES_CM_EVENT_RESET: + nes_debug(NES_DBG_CM, "cm_node = %p CM Event: RESET\n", + event->cm_node); + cm_event_reset(event); + break; + case NES_CM_EVENT_CONNECTED: + if ((!event->cm_node->cm_id) || + (event->cm_node->state != NES_CM_STATE_TSA)) break; - case NES_CM_EVENT_ABORTED: - if ((!event->cm_node->cm_id) || (event->cm_node->state == NES_CM_STATE_TSA)) { - break; - } - cm_event_connect_error(event); - nes_debug(NES_DBG_CM, "CM Event: ABORTED\n"); - break; - case NES_CM_EVENT_DROPPED_PKT: - nes_debug(NES_DBG_CM, "CM Event: DROPPED PKT\n"); - break; - default: - nes_debug(NES_DBG_CM, "CM Event: UNKNOWN EVENT TYPE\n"); + cm_event_connected(event); + nes_debug(NES_DBG_CM, "CM Event: CONNECTED\n"); + break; + case NES_CM_EVENT_ABORTED: + if ((!event->cm_node->cm_id) || + (event->cm_node->state == NES_CM_STATE_TSA)) break; + cm_event_connect_error(event); + nes_debug(NES_DBG_CM, "CM Event: ABORTED\n"); + break; + case NES_CM_EVENT_DROPPED_PKT: + nes_debug(NES_DBG_CM, "CM Event: DROPPED PKT\n"); + break; + default: + nes_debug(NES_DBG_CM, "CM Event: UNKNOWN EVENT TYPE\n"); + break; } atomic_dec(&cm_core->events_posted); diff --git a/drivers/infiniband/hw/nes/nes_cm.h b/drivers/infiniband/hw/nes/nes_cm.h index 7717cb2..367b3d2 100644 --- a/drivers/infiniband/hw/nes/nes_cm.h +++ b/drivers/infiniband/hw/nes/nes_cm.h @@ -83,6 +83,8 @@ enum nes_timer_type { #define SET_FIN 4 #define SET_RST 8 +#define TCP_OPTIONS_PADDING 3 + struct option_base { u8 optionnum; u8 length; @@ -177,6 +179,7 @@ enum nes_cm_node_state { NES_CM_STATE_ESTABLISHED, NES_CM_STATE_ACCEPTING, NES_CM_STATE_MPAREQ_SENT, + NES_CM_STATE_MPAREQ_RCVD, NES_CM_STATE_TSA, NES_CM_STATE_FIN_WAIT1, NES_CM_STATE_FIN_WAIT2, @@ -187,6 +190,16 @@ enum nes_cm_node_state { NES_CM_STATE_CLOSED }; +enum nes_tcpip_pkt_type { + NES_PKT_TYPE_UNKNOWN, + NES_PKT_TYPE_SYN, + NES_PKT_TYPE_SYNACK, + NES_PKT_TYPE_ACK, + NES_PKT_TYPE_FIN, + NES_PKT_TYPE_RST +}; + + /* type of nes connection */ enum nes_cm_conn_type { NES_CM_IWARP_CONN_TYPE, @@ -257,7 +270,9 @@ struct nes_cm_node { struct net_device *netdev; struct nes_cm_node *loopbackpartner; - struct list_head retrans_list; + + struct nes_timer_entry *send_entry; + spinlock_t retrans_list_lock; struct list_head recv_list; spinlock_t recv_list_lock; @@ -276,6 +291,8 @@ struct nes_cm_node { struct nes_vnic *nesvnic; int apbvt_set; int accept_pend; + int freed; + struct nes_qp *nesqp; }; /* structure for client or CM to fill when making CM api calls. */ @@ -366,14 +383,14 @@ struct nes_cm_ops { struct nes_cm_info *); int (*stop_listener)(struct nes_cm_core *, struct nes_cm_listener *); struct nes_cm_node * (*connect)(struct nes_cm_core *, - struct nes_vnic *, struct ietf_mpa_frame *, + struct nes_vnic *, u16, void *, struct nes_cm_info *); int (*close)(struct nes_cm_core *, struct nes_cm_node *); int (*accept)(struct nes_cm_core *, struct ietf_mpa_frame *, struct nes_cm_node *); int (*reject)(struct nes_cm_core *, struct ietf_mpa_frame *, struct nes_cm_node *); - int (*recv_pkt)(struct nes_cm_core *, struct nes_vnic *, + void (*recv_pkt)(struct nes_cm_core *, struct nes_vnic *, struct sk_buff *); int (*destroy_cm_core)(struct nes_cm_core *); int (*get)(struct nes_cm_core *); From rdreier at cisco.com Tue Jul 22 19:59:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 19:59:05 -0700 Subject: [ofa-general] Quetions about IPOIB handle last WQE event In-Reply-To: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Tue, 22 Jul 2008 17:21:22 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: [ ... IB spec stuff about last WQE reached events ... ] > The IPoIB-CM implementation takes the approach by posting another WR > that completes on the same CQ and wait for this WR to return as a WC. > IPoIB first puts the QP in error status, then waits for last WQE event > in async event handler by posting a drain WR, the QP resources will be > released in when last CEQs being generated. However it works for > ConnectionX but not for ehca. > > In ehca implemention it follows Section 11-5.2.5, when the QP gets in > the Error state, and there are no more WQEs on the RQ. So these QP > resources are never being released thus causes QP resources leakage, no > QPs can't be released at all. So when the maxium QPs are reached > (default nonSRQ is 128, SRQ is 4K), no more new connections can be > built. Nodes can't even be reachable. I don't understand what the problem is for ehca. Once the QP is in the error state, posting a WR to the send queue should complete immediately with a flush error status, and that completion should trigger IPoIB to clean up the QP. What goes wrong with ehca? > LAST WQE reached event for RX QP0 > post last WR for QP0 > poll_cq > below only applies to Mellanox, ehca won't see > last WQ in SRQ > ---------------- > see last WR for QP0 So you're saying that the send request doesn't complete for ehca? That seems like it must be a bug somewhere in the ehca driver/firmware/hardware. This has nothing to do with SRQ or last WQE reached events-- it is the basic requirement that send requests posted when a QP is in the error state complete with a flush error status. > Since nonSRQ doesn't handle async event, it never releases QPs, 128 > connections will run out soon even in a two nodes cluster by repeating > above steps. ( This is another bug, I will submit a fix). Yes, if non-SRQ doesn't free QPs, then this is another bug. > 2. If node-1 fails to send DREQ for any reason to remote, like node-1 > shutdown, then RX QP in node-2 will be put in the error list after > around 21 mins > (IPOIB_CM_RX_TIMEOUT + IPOIB_CM_RX_DELAY 5*256*HZ) > #define IPOIB_CM_RX_TIMEOUT (2 * 256 * HZ) > #define IPOIB_CM_RX_DELAY (3 * 256 * HZ)) > The timer seems too long for release stale QP resources, we could hit QP > run out in a large cluster even for mthca/mlx4. It is a long timeout, but how often does this case happen? When a node crashes? > 1. Whether it's a MUST to put QP in error status before posting last WR? > if it's a MUST, why? Yes, it's a must because we don't wnat the send executed, we want it to complete with an error status. > 2. Last WQE event is only generated once for each QP even IPoIB sets QP > into error status and the CI surfaced a Local Work Queue Catastrophic > Error on the same QP at the same time, is that right? Umm, a local work queue catastrophic error means something went wrong in the driver/firmware/hardware -- a consumer shouldn't be able to cause this type of event. Finding out why this catastrophic error happens should help debug things. - R. From mashirle at us.ibm.com Tue Jul 22 21:30:31 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Tue, 22 Jul 2008 21:30:31 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> Thanks Roland for your prompt reply. On Tue, 2008-07-22 at 19:59 -0700, Roland Dreier wrote: > I don't understand what the problem is for ehca. Once the QP is in the > error state, posting a WR to the send queue should complete immediately > with a flush error status, and that completion should trigger IPoIB to > clean up the QP. What goes wrong with ehca? > > LAST WQE reached event for RX QP0 > > post last WR for QP0 > > poll_cq > > below only applies to Mellanox, ehca won't see > > last WQ in SRQ > > ---------------- > > see last WR for QP0 > > So you're saying that the send request doesn't complete for ehca? That > seems like it must be a bug somewhere in the ehca > driver/firmware/hardware. This has nothing to do with SRQ or last WQE > reached events-- it is the basic requirement that send requests posted > when a QP is in the error state complete with a flush error status. Section: 11-5.2.5: ------------------ If the HCA supports SRQ, for RC and UD service, the CI shall generate a Last WQE Reached Affiliated Asynchronous Event on a QP that is in the Error State and is associated with an SRQ when either: • a CQE is generated for the last WQE, or • the QP gets in the Error State and there are no more WQEs on the RQ. I thought the ehca follows this "the QP gets in the Error State and there are no more WQEs on the RQ." so a CQE is not generated for the last WQE. > > Since nonSRQ doesn't handle async event, it never releases QPs, 128 > > connections will run out soon even in a two nodes cluster by repeating > > above steps. ( This is another bug, I will submit a fix). > > Yes, if non-SRQ doesn't free QPs, then this is another bug. > > > 2. If node-1 fails to send DREQ for any reason to remote, like node-1 > > shutdown, then RX QP in node-2 will be put in the error list after > > around 21 mins > > (IPOIB_CM_RX_TIMEOUT + IPOIB_CM_RX_DELAY 5*256*HZ) > > #define IPOIB_CM_RX_TIMEOUT (2 * 256 * HZ) > > #define IPOIB_CM_RX_DELAY (3 * 256 * HZ)) > > > The timer seems too long for release stale QP resources, we could hit QP > > run out in a large cluster even for mthca/mlx4. For a 2K nodes cluster, each node has 2 ports, IPoIB-CM maxium RX QPs are 4K, each node has 2 ports, even one node has a crash, the connection will be lost for 21 mins for that node after reboot. > It is a long timeout, but how often does this case happen? When a node > crashes? > > > 1. Whether it's a MUST to put QP in error status before posting last WR? > > if it's a MUST, why? > > Yes, it's a must because we don't wnat the send executed, we want it to > complete with an error status. In this case, the send side TX QP is already gone. So there shouldn't be any send executed from remote to this RX QP. And there is no harmful to deliver any outstanding CQEs in CQ to consumer. So it's OK not putting QP in error status before posting last WR, right? Any IB spec specifies somewhere it's a MUST? > > 2. Last WQE event is only generated once for each QP even IPoIB sets QP > > into error status and the CI surfaced a Local Work Queue Catastrophic > > Error on the same QP at the same time, is that right? > > Umm, a local work queue catastrophic error means something went wrong in > the driver/firmware/hardware -- a consumer shouldn't be able to cause > this type of event. Finding out why this catastrophic error happens > should help debug things. Sorry my question was not clear. It should be whether it's possible to have two last WQE reached event on the same QP: one is from consumer setting the QP to error status, one is from driver/FW/HW catastrophic error if any? Thanks Shirley From rdreier at cisco.com Tue Jul 22 21:57:26 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 22 Jul 2008 21:57:26 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Tue, 22 Jul 2008 21:30:31 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > Section: 11-5.2.5: > ------------------ > If the HCA supports SRQ, for RC and UD service, the CI shall generate a > Last WQE Reached Affiliated Asynchronous Event on a QP that is in the > Error State and is associated with an SRQ when either: > • a CQE is generated for the last WQE, or > • the QP gets in the Error State and there are no more WQEs on the > RQ. > > I thought the ehca follows this "the QP gets in the Error State and > there are no more WQEs on the RQ." so a CQE is not generated for the > last WQE. I don't follow -- we transition the QP to error. The HCA generates a "last WQE reached" event. Then we post a send request to the QP (which is now in the error state). This send request must complete with a flush error status. There's nothing tricky about this, and there's nothing related to RQ/SRQ processing or the last WQE reached event. If ehca does not complete send requests posted to a QP in the error state with a flush error, then there is a bug somewhere in the ehca driver/firmware/hardware. > In this case, the send side TX QP is already gone. So there shouldn't be > any send executed from remote to this RX QP. And there is no harmful to > deliver any outstanding CQEs in CQ to consumer. So it's OK not putting > QP in error status before posting last WR, right? Any IB spec specifies > somewhere it's a MUST? I guess it might work, but how do we avoid leaking receive work requests if we don't transition the QP to error at some point? There are cases where we might be garbage collecting an unused connection and race with an incoming message. Why wouldn't we want to make everything simple and transition to the error state? > Sorry my question was not clear. It should be whether it's possible to > have two last WQE reached event on the same QP: one is from consumer > setting the QP to error status, one is from driver/FW/HW catastrophic > error if any? A catastrophic error should be reported as a catastrophic error, not a last WQE reached event. Of course it is possible to get multiple last WQE reached events for a single QP, by transitioning the QP out of the error state and then back into the error state. But IPoIB wouldn't do that. - R. From mashirle at us.ibm.com Tue Jul 22 23:01:06 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Tue, 22 Jul 2008 23:01:06 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> On Tue, 2008-07-22 at 21:57 -0700, Roland Dreier wrote: > If ehca does not complete > send requests posted to a QP in the error state with a flush error, then > there is a bug somewhere in the ehca driver/firmware/hardware. Got it, will discuss this with driver team. > > In this case, the send side TX QP is already gone. So there shouldn't be > > any send executed from remote to this RX QP. And there is no harmful to > > deliver any outstanding CQEs in CQ to consumer. So it's OK not putting > > QP in error status before posting last WR, right? Any IB spec specifies > > somewhere it's a MUST? > > I guess it might work, but how do we avoid leaking receive work requests > if we don't transition the QP to error at some point? There are cases > where we might be garbage collecting an unused connection and race with > an incoming message. Why wouldn't we want to make everything simple and > transition to the error state? I agree that the current approach is pretty simple by using ipoib_cm_rx_event_handler() to handle QP resources release in all cases: connections set up, established, staled for SRQ. What's in my mind was to use a common approach for both nonSRQ and SRQ. Because nonSRQ doesn't have last WQE, but we could do post_send last WR when receiving DREQ to handle QP resource destroy for both nonSRQ and SRQ without setting QP to error status. For stale connections, like not active for a while, remote QP node has died for any reasons: shutdown or crash, we can put the QP in error status, then after a certain timer (CQEs has been processed already), we can destroy the QP. For QP hasn't been in RTU status yet, like REJ received, we can release QP resource immediately since the QP is not ready to accept any incoming messages yet. How do you think? thanks Shirley From retest at isquarer.com Wed Jul 23 00:33:16 2008 From: retest at isquarer.com (Blair Jackson) Date: Wed, 23 Jul 2008 07:33:16 -0000 Subject: [ofa-general] Office 2008 Special Media Edition Message-ID: <000801c8ec95$51664c00$0100007f@aikvee> 'The following titles and more are now available for pcs and macs* CyberLink PowerDVD ULTRA EDITION v8 Office 2008 Special Media Edition Traktor DJ Studio Office 2008 Special Media Edition Office 2007 Ultimate Windows Home Server ' softsavingsnow . com * System Requirements * For PC: * Intel Pentium 4 (1.4GHz processor), Intel Centrino, Intel Xeon Office 2007 Ultimate or Intel Core Duo (or Office 2007 Ultimate) processor; SSE2-enabled processor required for AMD systems & Microsoft Windows XP with Service Pack 2 or Microsoft Windows Vista Home Premium, Business, Ultimate, or Enterprise (certified for 32-bit editions) * 512MB of RAM or more * 1GB of available hard-disk space (additional free space Office 2007 Ultimate) * Microsoft compatible sound card (multichannel ASIO-compatible sound card recommended) * 1,024x768 monitor resolution with 32-bit color adapter recommended * CD-R or DVD-ROM drive ' For MAC: ' PowerPC G4 or G5 or multicore Intel processor ' Mac OS X or similar ' 512MB of RAM or more ' 1GB of available hard-disk space ' Core Audio compatible sound card ' 1,024x768 monitor resolution with 32-bit color adapter ' DVD-ROM drive* DVD+-R burner required for DVD creation On today's "Africa Update," Zimbabwe's leaders agree to hold talks to resolve the country's political crisis. Plus, the United States lifts a ban on visitors with HIV, and we take a look at how Africa's oil affects the cost of gas in America. From Robert at saq.co.uk Wed Jul 23 01:48:11 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Wed, 23 Jul 2008 09:48:11 +0100 Subject: [ofa-general] OFED build problems on Centos 5.2 Message-ID: I'm trying to compile OFED 1.3.1 on Centos 5.2 (The Xen Kernel). I have installed all the dependencies listed in readme.txt (And a few extra including kernel-devel-xen). I cannot get it to install though, below is the errors from the log, I'm guessing it is probably a dependency problem or maybe an existing OFED install interfering but really cannot find what is needed. I am quite new to this but have managed an OFED install on a couple of different Centos 5.1 systems. Does anyone know how to fix this? Thanks in advance, Rob + test -e /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/infini band/qlgc_vnic.ko + INFO=/var/tmp/OFED/etc/infiniband/info + /bin/rm -f /var/tmp/OFED/etc/infiniband/info + mkdir -p /var/tmp/OFED/etc/infiniband + touch /var/tmp/OFED/etc/infiniband/info + cat + chmod +x /var/tmp/OFED/etc/infiniband/info + install -d /var/tmp/OFED//etc/infiniband + install -m 0644 /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3.1/ofed_scripts/openib.conf /var/tmp/OFED//etc/infiniband + install -d /var/tmp/OFED/etc/init.d + install -m 0755 /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3.1/ofed_scripts/openibd /var/tmp/OFED/etc/init.d + install -d /var/tmp/OFED/etc/udev/rules.d + install -m 0644 /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3.1/ofed_scripts/90-ib.rules /var/tmp/OFED/etc/udev/rules.d + case "$(udevinfo -V 2> /dev/null | awk '{print $NF}' 2> /dev/null)" in ++ udevinfo -V ++ awk '{print $NF}' + /usr/lib/rpm/redhat/brp-compress + /usr/lib/rpm/redhat/brp-strip /usr/bin/strip + /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip + /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump + /usr/lib/rpm/brp-python-bytecompile + /usr/lib/rpm/redhat/brp-java-repack-jars Processing files: kernel-ib-1.3.1-2.6.18_92.el5xen error: File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/i scsi_tcp.ko error: File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/l ibiscsi.ko error: File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/s csi_transport_iscsi.ko error: File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/net/rds error: File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/cx gb3 error: File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/ml x4 Processing files: kernel-ib-devel-1.3.1-2.6.18_92.el5xen Requires(interp): /bin/sh /bin/sh /bin/sh Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 Requires(pre): /bin/sh Requires(post): /bin/sh Requires(postun): /bin/sh Requires: /bin/bash kernel-ib RPM build errors: user vlad does not exist - using root group vlad does not exist - using root user vlad does not exist - using root group vlad does not exist - using root File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/i scsi_tcp.ko File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/l ibiscsi.ko File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/s csi_transport_iscsi.ko File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/net/rds File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/cx gb3 File not found: /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/ml x4 The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Wed Jul 23 02:54:00 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Wed, 23 Jul 2008 02:54:00 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080723-0200 daily build status Message-ID: <20080723095400.A854CE60C2D@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Failed: Build failed on ia64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.21.1_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.21.1' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.22_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.22' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.26 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.26_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.26' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.25 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.25_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.18-8.el5_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.18-8.el5' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.24 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.24_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.24' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:316: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:324: warning: assignment makes pointer from integer without a cast /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c: In function 'rds_ib_conn_shutdown': /home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.c:750: error: implicit declaration of function 'vfree' make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check/net/rds/ib_cm.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check/net/rds] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080723-0200_linux-2.6.19_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From xwu4 at connect.carleton.ca Wed Jul 23 03:42:16 2008 From: xwu4 at connect.carleton.ca (Xianguo Wu) Date: Wed, 23 Jul 2008 06:42:16 -0400 (EDT) Subject: [ofa-general] Ticket Number is 05-08-10-18-20-46-{43} Message-ID: <8128822.1216809736444.JavaMail.xwu4@connect.carleton.ca> Attn: Winner, You have won the sum of £850,000.00 British Pounds Sterling from Online Sweepstake Bonaza for the month of Jully. Your Ticket Number is 05-08-10-18-20-46-{43} To further the claims of your prize, you are advise to contact your claims department with the below informations. Contact Person: Dr. Brand Parker Executive Secretary/Claims agent Email: brand.parker at live.com Tell: +44-702-402-2030 Claims Requirements: 1.full name: 2.Home Address: 3.Age: 4.Sex: 5.Marital Status: 7.Phone Number: 8.Nationality, Occupation. 9.Country Of Residence. Regards, Xianguo Wu Head Customercare Service The British Lottery Company From vlad at mellanox.co.il Wed Jul 23 04:25:16 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Wed, 23 Jul 2008 14:25:16 +0300 Subject: [ofa-general] OFED build problems on Centos 5.2 In-Reply-To: References: Message-ID: <4887151C.3000203@mellanox.co.il> Robert Dunkley wrote: > I’m trying to compile OFED 1.3.1 on Centos 5.2 (The Xen Kernel). I have > installed all the dependencies listed in readme.txt (And a few extra > including kernel-devel-xen). I cannot get it to install though, below is > the errors from the log, I’m guessing it is probably a dependency > problem or maybe an existing OFED install interfering but really cannot > find what is needed. I am quite new to this but have managed an OFED > install on a couple of different Centos 5.1 systems. Does anyone know > how to fix this? > > Thanks in advance, > > Rob > > error: File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/iscsi_tcp.ko > error: File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/libiscsi.ko > error: File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/scsi_transport_iscsi.ko > error: File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/net/rds > error: File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/cxgb3 > error: File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/mlx4 > Processing files: kernel-ib-devel-1.3.1-2.6.18_92.el5xen > Requires(interp): /bin/sh /bin/sh /bin/sh > Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 > rpmlib(PayloadFilesHavePrefix) <= 4.0-1 > Requires(pre): /bin/sh > Requires(post): /bin/sh > Requires(postun): /bin/sh > Requires: /bin/bash kernel-ib > > RPM build errors: > user vlad does not exist - using root > group vlad does not exist - using root > user vlad does not exist - using root > group vlad does not exist - using root > > File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/iscsi_tcp.ko > File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/libiscsi.ko > File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/scsi/scsi_transport_iscsi.ko > File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/net/rds > File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/cxgb3 > File not found: > /var/tmp/OFED/lib/modules/2.6.18-92.el5xen/updates/kernel/drivers/net/mlx4 > Hi Robert, From the log file you send I see that kernel modules were installed to some different place. Please run the following command and send me its output and the full compilation log file (ofa_kernel.rpmbuild.log). find /var/tmp -name '*.ko' Regards, Vladimir From kliteyn at dev.mellanox.co.il Wed Jul 23 04:29:13 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 23 Jul 2008 14:29:13 +0300 Subject: [ofa-general] OpenSM "Dead end on path to LID" In-Reply-To: <4886278B.2080502@noaa.gov> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> <4880C61A.6010000@noaa.gov> <48811106.9030906@dev.mellanox.co.il> <4886278B.2080502@noaa.gov> Message-ID: <48871609.5020403@dev.mellanox.co.il> Nathan Dauchy wrote: >>> I believe MAXSMPS was originally set to 0 (unlimited), based on >>> duplicating the config file from an older SM setup. I reduced it to 32 >>> when we saw some IB errors on standalone System B. I'm afraid I don't >>> have documentation on what those problems were, but I don't recall >>> seeing the exact same symptoms. I think it was MAD timeout error >>> messages that prompted me to change the MAXSMPS value. >> You probably have the timeout errors in the osm log now as well. >> These errors are followed by the "unknown remote side" messages, >> which means that osm didn't get response to some MADs (timeout). >> >>> We will be able to test any fixes during a scheduled system downtime on >>> 7/24. At that point, do you recommend trying MAXSMPS=4? >> In ofed 1.3.1, the default conf file value for maxsmps should be 4. >> If you're using 0, then I'm pretty sure that this is what causing >> the problems. Try 4 - it should be fine. > > We went ahead and tried both MAXSMPS=4 and MAXSMPS=1. The symptoms did > not improve with all the nodes booted. :( > > For the record, here is exactly how opensm is running now: > > # ps uaxw | grep open > root 23112 1.0 0.1 288432 16732 ? Sl 18:10 0:07 > /opt/ofed/1.3.1/sbin/opensm -maxsmps 1 -t 600 -f /var/run/osm/osm.log -R > updn -g 0 --honor_guid2lid Please try running opensm with a default routing (w/o '-R updn'). Just trying to understand if this is a routing or discovery issue. Also, where does the opensm run? -- Yevgeny >>> If that doesn't work, what else should we be prepared to try, or what >>> other debugging information would be helpful to gather? >>> >>> For the record, other steps we are considering: >>> * Latest OpenSM code >>> * Upgrading firmware on all IB switches >>> * Changing topology to remove the 12X links (ugh!) >> Hope that there won't be a need for other steps. > > To try to take the 12X links out of the equation, we replaced the (3) > "Subtree C" spine switches with ones burned with the default 24*4X .INI > file, and removed 2 cables from each 3-cable bundle. The "Subtree C" > Clos/Edge switches and Root/Aggregation switches were still configured > as 12X links, but according to "ibdiagnet", they negotiated down to 4X. > > (I can furnish a nicer diagram of our IB tree off-list if anyone would > like to take a look at it.) > > > Any other things we can try? > > > Thanks again, > Nathan > From ogerlitz at voltaire.com Wed Jul 23 05:42:11 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 23 Jul 2008 15:42:11 +0300 Subject: [ofa-general] Re: [PATCH v2 3/4] rdma/cma: implement RDMA_CM_EVENT_ADDR_CHANGE notification In-Reply-To: References: Message-ID: <48872723.80609@voltaire.com> Roland Dreier wrote: > thanks, I applied this. Do you refer to your infiniband tree at kernel.org? I don't see this patch being in the for-next nor in the for-2.6.27 branches Or. From jackm at dev.mellanox.co.il Wed Jul 23 07:16:26 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 23 Jul 2008 17:16:26 +0300 Subject: [ofa-general] OFED 1.4 XRC implementation: XRC apps will need to recompile at 1.4 alpha release (or a bit sooner) Message-ID: <200807231716.26571.jackm@dev.mellanox.co.il> Users of XRC under OFED 1.3 users will need to recompile when OFED 1.4 gets released, due to an ABI alignment bug that Roland spotted (for x86 libraries on x86_64 machines). I'll not be fixing the bug in OFED 1.3, to avoid confusion (no one has complained of this bug as yet). However, when some user tries 32-bit libraries on a 64 bit machine, XRC may not work (due to the alignment bug). I'll be adding the fix to OFED 1.4 (daily build) once the upstream kernel XRC implementation stabilizes. I'll send mail again to alert you when I do this. - Jack From weiny2 at llnl.gov Wed Jul 23 08:03:39 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 23 Jul 2008 08:03:39 -0700 Subject: [ofa-general] [PATCH V2] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: References: <20080722172809.5d0831f3.weiny2@llnl.gov> Message-ID: <20080723080339.73f3914f.weiny2@llnl.gov> On Tue, 22 Jul 2008 21:06:40 -0400 "Hal Rosenstock" wrote: > One nit below... > > On Tue, Jul 22, 2008 at 8:28 PM, Ira Weiny wrote: > > >From be896a92f5e78fc315b4dbfcb77e7d8f07612769 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Tue, 22 Jul 2008 16:31:36 -0700 > > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM > > > > > > + case 144: > > + return ("CapabilityMask, NodeDescriptor, Link [Width|Speed] > > ^^^^^^^^^^^ > > Description > Enabled changed"); > I have always called it that?!?!? I may never learn... V2 attached below. Ira >From 013a5027ac2f9ffd01569618aac14cecb54e5428 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 22 Jul 2008 16:31:36 -0700 Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 43 +++++++++++++++++++++++++++++++++++++++++ opensm/opensm/osm_inform.c | 3 +- opensm/opensm/osm_trap_rcv.c | 10 +++++++- 3 files changed, 53 insertions(+), 3 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 09ec257..8d618f3 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7219,6 +7219,49 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) +static inline char * OSM_API +ib_trap_str(uint16_t trap_num) +{ + switch(trap_num) + { + case 64: + return ("GID in service"); + case 65: + return ("GID out of service"); + case 66: + return ("New mcast group created"); + case 67: + return ("Mcast group deleted"); + case 68: + return ("UnPath, Path no longer valid"); + case 69: + return ("RePath, Path recomputed"); + case 128: + return ("Link state change"); + case 129: + return ("Local Link integrity threshold reached"); + case 130: + return ("Excessive Buffer Overrun Threshold reached"); + case 131: + return ("Flow Control Update watchdog timer expired"); + case 144: + return ("CapabilityMask, NodeDescription, Link [Width|Speed] Enabled changed"); + case 145: + return ("System Image GUID changed"); + case 256: + return ("Bad M_Key"); + case 257: + return ("Bad P_Key"); + case 258: + return ("Bad Q_Key"); + case 259: + return ("Bad P_Key (switch external port)"); + default: + break; + } + return ("Unknown"); +} + /****f* IBA Base: Types/ib_notice_is_generic * NAME * ib_notice_is_generic diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index e6146f7..77dad32 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, /* an official Event information log */ if (ib_notice_is_generic(p_ntc)) { OSM_LOG(p_log, OSM_LOG_INFO, - "Reporting Generic Notice type:%u num:%u" + "Reporting Generic Notice type:%u num:%u (%s)" " from LID:%u GID:0x%016" PRIx64 ",0x%016" PRIx64 "\n", ib_notice_get_type(p_ntc), cl_ntoh16(p_ntc->g_or_v.generic.trap_num), + ib_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), cl_ntoh16(p_ntc->issuer_lid), cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index ae7ab05..bcb4260 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, || (p_ntci->g_or_v.generic.trap_num == CL_HTON16(131))) OSM_LOG(sm->p_log, OSM_LOG_ERROR, - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " + "Received Generic Notice type:0x%02X " + "num:%u (%s) Producer:%u (%s) " "from LID:%u Port %d TID:0x%016" PRIx64 "\n", ib_notice_get_type(p_ntci), cl_ntoh16(p_ntci->g_or_v.generic. trap_num), + ib_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. + trap_num)), cl_ntoh32(ib_notice_get_prod_type (p_ntci)), ib_get_producer_type_str @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, ); else OSM_LOG(sm->p_log, OSM_LOG_ERROR, - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " + "Received Generic Notice type:0x%02X " + "num:%u (%s) Producer:%u (%s) " "from LID:%u TID:0x%016" PRIx64 "\n", ib_notice_get_type(p_ntci), cl_ntoh16(p_ntci->g_or_v.generic. trap_num), + ib_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. + trap_num)), cl_ntoh32(ib_notice_get_prod_type (p_ntci)), ib_get_producer_type_str -- 1.5.4.5 From yossi.openib at gmail.com Wed Jul 23 08:23:25 2008 From: yossi.openib at gmail.com (Yosef Eitgin) Date: Wed, 23 Jul 2008 18:23:25 +0300 Subject: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: References: <48849F59.7060502@Voltaire.COM> Message-ID: <48874CED.5020307@gmail.com> Roland Dreier wrote: > > The problem: > > A multicast sender joins the MGID as full member, but does not leave > > (as long as the interface is up). This causes an MGID leakage in the > > SM. > > I think ipoib has used send-only joins for a while now. But the problem > of never cleaning up these memberships stands. The problem worse because ipoib is not really using send-only joins, it joins as full member. See ipoib_multicast.c/ipoib_mcast_sendonly_join(): struct ib_sa_mcmember_rec rec = { #if 0 /* Some SMs don't support send-only yet */ .join_state = 4 #else .join_state = 1 #endif }; > > Thanks for looking at this -- it would be nice to deal with this. > > > An issue we found with this solution, is that it takes a while until the > > neighbour is cleaned, and it requires that the host will send some traffic > > (I supposed that unless we put more entries in the routing table, out neighbour > > will not be garbage collected). > > Would it work to have a more naive approach, and just keep track of the > last time each send-only membership was used, and set up a timer that > runs, say every minute, and garbage collect send-only memberships that > are more than 2 minutes old, or something like that? I guess it would, and it will sure be simpler. > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From rdreier at cisco.com Wed Jul 23 09:16:48 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 23 Jul 2008 09:16:48 -0700 Subject: [ofa-general] Re: [PATCH v2 3/4] rdma/cma: implement RDMA_CM_EVENT_ADDR_CHANGE notification In-Reply-To: <48872723.80609@voltaire.com> (Or Gerlitz's message of "Wed, 23 Jul 2008 15:42:11 +0300") References: <48872723.80609@voltaire.com> Message-ID: I may have forgotten to push. It should be in for-next at least. From rdreier at cisco.com Wed Jul 23 09:20:39 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 23 Jul 2008 09:20:39 -0700 Subject: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: <48874CED.5020307@gmail.com> (Yosef Eitgin's message of "Wed, 23 Jul 2008 18:23:25 +0300") References: <48849F59.7060502@Voltaire.COM> <48874CED.5020307@gmail.com> Message-ID: > The problem worse because ipoib is not really using send-only joins, it > joins as full member. See ipoib_multicast.c/ipoib_mcast_sendonly_join(): > > struct ib_sa_mcmember_rec rec = { > #if 0 /* Some SMs don't support send-only yet */ > .join_state = 4 > #else > .join_state = 1 > #endif > }; Sorry, you're right. I misread that code. I guess the time has come to use send-only joins... I think all relevant SMs have supported it for a long time. - R. From yossi.openib at gmail.com Wed Jul 23 09:30:14 2008 From: yossi.openib at gmail.com (Yossi Etigin) Date: Wed, 23 Jul 2008 19:30:14 +0300 Subject: ***SPAM*** Re: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: References: <48849F59.7060502@Voltaire.COM> <48874CED.5020307@gmail.com> Message-ID: <48875C96.70006@gmail.com> AFAIK OpenSM does not really support send-only, but it uses NonMember to emulate it. Roland Dreier wrote: > > The problem worse because ipoib is not really using send-only joins, it > > joins as full member. See ipoib_multicast.c/ipoib_mcast_sendonly_join(): > > > > struct ib_sa_mcmember_rec rec = { > > #if 0 /* Some SMs don't support send-only yet */ > > .join_state = 4 > > #else > > .join_state = 1 > > #endif > > }; > > Sorry, you're right. I misread that code. > > I guess the time has come to use send-only joins... I think all relevant > SMs have supported it for a long time. > > - R. > From Nathan.Dauchy at noaa.gov Wed Jul 23 09:57:17 2008 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Wed, 23 Jul 2008 10:57:17 -0600 Subject: [ofa-general] OpenSM "Dead end on path to LID" -- problem with updn, fixed with minhop In-Reply-To: <48871609.5020403@dev.mellanox.co.il> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> <4880C61A.6010000@noaa.gov> <48811106.9030906@dev.mellanox.co.il> <4886278B.2080502@noaa.gov> <48871609.5020403@dev.mellanox.co.il> Message-ID: <488762ED.1010307@noaa.gov> Yevgeny Kliteynik wrote: > Nathan Dauchy wrote: >> We went ahead and tried both MAXSMPS=4 and MAXSMPS=1. The symptoms did >> not improve with all the nodes booted. :( >> >> For the record, here is exactly how opensm is running now: >> >> # ps uaxw | grep open >> root 23112 1.0 0.1 288432 16732 ? Sl 18:10 0:07 >> /opt/ofed/1.3.1/sbin/opensm -maxsmps 1 -t 600 -f /var/run/osm/osm.log -R >> updn -g 0 --honor_guid2lid > > Please try running opensm with a default routing (w/o '-R updn'). > Just trying to understand if this is a routing or discovery issue. Yevgeny, thanks for your continued help! I modified the /etc/ofa/opensm.conf file to include: UPDN="off" And then restarted opensm. Here is what it looks like now: [root at wms4 ~]# ps uaxw | grep open root 28360 0.9 0.1 280756 16724 pts/3 Sl 16:20 0:06 /opt/ofed/1.3.1/sbin/opensm -maxsmps 1 -t 600 -f /var/run/osm/osm.log -g 0 --honor_guid2lid So far, the results look good!!! We now have all hosts booted, communicating over IB, and "ibdiagnet" finishes cleanly. Given the network topology I described previously, how is "minhop" expected to behave differently than "updn"? Unless you recommend otherwise, I now hope to add back in the missing features and make sure everything is still OK: LMC=2 MAXSMPS=4 TIMEOUT=600 12X links on subtree C. > > Also, where does the opensm run? > For the record, the SM host is connected to one of the "Subtree B" edge switches. Hardware is an 8-core Intel Xeon 5450 @ 3.00GHz with 16GB RAM, and a single-port DDR "MT25204" HCA. Software is CentOS-5.1, linux-2.6.18-53.1.21.el5, and OFED-1.3.1. Please let me know if there is additional diagnostic information I can gather in order to create a test case for future improvements to the "updn" routing engine. Thanks again, Nathan From olga.shern at gmail.com Wed Jul 23 10:13:30 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Wed, 23 Jul 2008 20:13:30 +0300 Subject: ***SPAM*** Re: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: <48875C96.70006@gmail.com> References: <48849F59.7060502@Voltaire.COM> <48874CED.5020307@gmail.com> <48875C96.70006@gmail.com> Message-ID: In addition to that, we have found more issues with this approach: According to IPoIB RFC if SEND ONLY join is used then; When multicast group doesn't exist then 2 things should be done: 1.Send multicast packets on all routers multicast group 2.Register to get multicast group creation / deletion notification traps from SM Therefore the disadvantage of this approach is: A) Traps mechanism is not scalable. B) It is complicated mechanism: If multicast group doesn't exist need to register to get trap from SM on group creation, meanwhile sending traffic on all routers group. Got trap from SM that multicast have been created, need to join it and also register for the trap of multicast group deletion. Start sending traffic on the original multicast group. If got trap that the group is deleted, need to start sending traffic on the all routers group …. Garbage collector approach is much simpler and more scalable. Olga P.S. Registration to group creation/deletion notification trap is not implemented in SA, the change that should be done is also big and complicated On 7/23/08, Yossi Etigin wrote: > AFAIK OpenSM does not really support send-only, but it uses NonMember to > emulate it. > > Roland Dreier wrote: > > > The problem worse because ipoib is not really using send-only joins, it > > > joins as full member. See > ipoib_multicast.c/ipoib_mcast_sendonly_join(): > > > > struct ib_sa_mcmember_rec rec = { > > > #if 0 /* Some SMs don't support > send-only yet */ > > > .join_state = 4 > > > #else > > > .join_state = 1 > > > #endif > > > }; > > > > Sorry, you're right. I misread that code. > > > > I guess the time has come to use send-only joins... I think all relevant > > SMs have supported it for a long time. > > > > - R. > > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From yossi.openib at gmail.com Wed Jul 23 11:22:02 2008 From: yossi.openib at gmail.com (Yossi Etigin) Date: Wed, 23 Jul 2008 21:22:02 +0300 Subject: ***SPAM*** Re: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: References: <48849F59.7060502@Voltaire.COM> Message-ID: <488776CA.6010001@gmail.com> > > Would it work to have a more naive approach, and just keep track of the > last time each send-only membership was used, and set up a timer that > runs, say every minute, and garbage collect send-only memberships that > are more than 2 minutes old, or something like that? > How about this one: diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index ca126fc..7a5998f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -88,6 +88,7 @@ enum { IPOIB_FLAG_ADMIN_CM = 9, IPOIB_FLAG_UMCAST = 10, IPOIB_FLAG_CSUM = 11, + IPOIB_MCAST_RUN_GC = 12, IPOIB_MAX_BACKOFF_SECONDS = 16, @@ -127,6 +128,7 @@ struct ipoib_mcast { struct list_head list; unsigned long created; + unsigned long used; unsigned long backoff; unsigned long flags; @@ -275,7 +277,8 @@ struct ipoib_dev_priv { struct rb_root multicast_tree; struct delayed_work pkey_poll_task; - struct delayed_work mcast_task; + struct delayed_work mcast_join_task; + struct delayed_work mcast_leave_task; struct work_struct flush_task; struct work_struct restart_task; struct delayed_work ah_reap_task; @@ -440,6 +443,7 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port); void ipoib_dev_cleanup(struct net_device *dev); void ipoib_mcast_join_task(struct work_struct *work); +void ipoib_mcast_leave_task(struct work_struct *work); void ipoib_mcast_send(struct net_device *dev, void *mgid, struct sk_buff *skb); void ipoib_mcast_restart_task(struct work_struct *work); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 2442090..b7f4b2b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -990,7 +990,8 @@ static void ipoib_setup(struct net_device *dev) INIT_DELAYED_WORK(&priv->pkey_poll_task, ipoib_pkey_poll); INIT_WORK(&priv->pkey_event_task, ipoib_pkey_event); - INIT_DELAYED_WORK(&priv->mcast_task, ipoib_mcast_join_task); + INIT_DELAYED_WORK(&priv->mcast_join_task, ipoib_mcast_join_task); + INIT_DELAYED_WORK(&priv->mcast_leave_task, ipoib_mcast_leave_task); INIT_WORK(&priv->flush_task, ipoib_ib_dev_flush); INIT_WORK(&priv->restart_task, ipoib_mcast_restart_task); INIT_DELAYED_WORK(&priv->ah_reap_task, ipoib_reap_ah); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 3f663fb..1957e93 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -120,6 +120,7 @@ static struct ipoib_mcast *ipoib_mcast_alloc(struct net_device *dev, mcast->dev = dev; mcast->created = jiffies; + mcast->used = jiffies; mcast->backoff = 1; INIT_LIST_HEAD(&mcast->list); @@ -389,7 +390,7 @@ static int ipoib_mcast_join_complete(int status, mutex_lock(&mcast_mutex); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) queue_delayed_work(ipoib_workqueue, - &priv->mcast_task, 0); + &priv->mcast_join_task, 0); mutex_unlock(&mcast_mutex); if (mcast == priv->broadcast) @@ -422,7 +423,7 @@ static int ipoib_mcast_join_complete(int status, mutex_lock(&mcast_mutex); spin_lock_irq(&priv->lock); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) - queue_delayed_work(ipoib_workqueue, &priv->mcast_task, + queue_delayed_work(ipoib_workqueue, &priv->mcast_join_task, mcast->backoff * HZ); spin_unlock_irq(&priv->lock); mutex_unlock(&mcast_mutex); @@ -492,7 +493,7 @@ static void ipoib_mcast_join(struct net_device *dev, struct ipoib_mcast *mcast, mutex_lock(&mcast_mutex); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) queue_delayed_work(ipoib_workqueue, - &priv->mcast_task, + &priv->mcast_join_task, mcast->backoff * HZ); mutex_unlock(&mcast_mutex); } @@ -501,7 +502,7 @@ static void ipoib_mcast_join(struct net_device *dev, struct ipoib_mcast *mcast, void ipoib_mcast_join_task(struct work_struct *work) { struct ipoib_dev_priv *priv = - container_of(work, struct ipoib_dev_priv, mcast_task.work); + container_of(work, struct ipoib_dev_priv, mcast_join_task.work); struct net_device *dev = priv->dev; if (!test_bit(IPOIB_MCAST_RUN, &priv->flags)) @@ -530,7 +531,7 @@ void ipoib_mcast_join_task(struct work_struct *work) mutex_lock(&mcast_mutex); if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) queue_delayed_work(ipoib_workqueue, - &priv->mcast_task, HZ); + &priv->mcast_join_task, HZ); mutex_unlock(&mcast_mutex); return; } @@ -591,7 +592,9 @@ int ipoib_mcast_start_thread(struct net_device *dev) mutex_lock(&mcast_mutex); if (!test_and_set_bit(IPOIB_MCAST_RUN, &priv->flags)) - queue_delayed_work(ipoib_workqueue, &priv->mcast_task, 0); + queue_delayed_work(ipoib_workqueue, &priv->mcast_join_task, 0); + if (!test_and_set_bit(IPOIB_MCAST_RUN_GC, &priv->flags)) + queue_delayed_work(ipoib_workqueue, &priv->mcast_leave_task, 0); mutex_unlock(&mcast_mutex); spin_lock_irq(&priv->lock); @@ -613,7 +616,9 @@ int ipoib_mcast_stop_thread(struct net_device *dev, int flush) mutex_lock(&mcast_mutex); clear_bit(IPOIB_MCAST_RUN, &priv->flags); - cancel_delayed_work(&priv->mcast_task); + clear_bit(IPOIB_MCAST_RUN_GC, &priv->flags); + cancel_delayed_work(&priv->mcast_join_task); + cancel_delayed_work(&priv->mcast_leave_task); mutex_unlock(&mcast_mutex); if (flush) @@ -720,6 +725,7 @@ out: } } + mcast->used = jiffies; ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN); } @@ -864,6 +870,33 @@ void ipoib_mcast_restart_task(struct work_struct *work) ipoib_mcast_start_thread(dev); } +void ipoib_mcast_leave_task(struct work_struct *work) +{ + struct ipoib_dev_priv *priv = + container_of(work, struct ipoib_dev_priv, mcast_leave_task.work); + struct net_device *dev = priv->dev; + struct ipoib_mcast *mcast, *tmcast; + LIST_HEAD(remove_list); + + if (!test_bit(IPOIB_MCAST_RUN_GC, &priv->flags)) + return; + + list_for_each_entry_safe(mcast, tmcast, &priv->multicast_list, list) { + if (test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags) && + time_before(mcast->used, jiffies - 120 * HZ)) { + rb_erase(&mcast->rb_node, &priv->multicast_tree); + list_move_tail(&mcast->list, &remove_list); + } + } + + list_for_each_entry_safe(mcast, tmcast, &remove_list, list) { + ipoib_mcast_leave(dev, mcast); + ipoib_mcast_free(mcast); + } + + queue_delayed_work(ipoib_workqueue, &priv->mcast_leave_task, 60 * HZ); +} + #ifdef CONFIG_INFINIBAND_IPOIB_DEBUG struct ipoib_mcast_iter *ipoib_mcast_iter_init(struct net_device *dev) From flatif at neteffect.com Wed Jul 23 12:18:54 2008 From: flatif at neteffect.com (Faisal Latif) Date: Wed, 23 Jul 2008 14:18:54 -0500 Subject: [ofa-general] ***SPAM*** [PATCH v2] RDMA/nes: CM connection setup/teardown rework Message-ID: <200807231918.m6NJIsBV009347@velma.neteffect.com> Major rework of CM connection setup/teardown. We had a number of issues with MPI applications not starting/terminating properly over time. With these changes we were able to run longer on larger clusters. * Removed memory allocation from nes_connect() and nes_cm_connect(). * Fixed mini_cm_dec_refcnt_listen() when destroying listener. * Removed unnecessary code from schedule_nes_timer() and nes_cm_timer_tick(). * Functionalized mini_cm_recv_pkt() and process_packet(). * Cleaned up cm_node->ref_count usage. * Reuse skbs if available. v2 - added missing reference counter changes to three files (nes.c, nes_hw.c and nes_verbs.c) Signed-off-by: Faisal Latif --- We have a few more CM changes planned for 2.6.27: * Modify nes_cm_timer_tick() to use separate timer pending cm_node list. * Modify async event handling in nes_process_iwarp_aeqe(). * Create pending listener list for netdev/CM node with invalid IPs. 5 files changed, 1156 insertions(+), 929 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes.c b/drivers/infiniband/hw/nes/nes.c index d2884e7..b0cab64 100644 --- a/drivers/infiniband/hw/nes/nes.c +++ b/drivers/infiniband/hw/nes/nes.c @@ -276,6 +276,7 @@ static void nes_cqp_rem_ref_callback(struct nes_device *nesdev, struct nes_cqp_r } nes_free_resource(nesadapter, nesadapter->allocated_qps, nesqp->hwqp.qp_id); + nesadapter->qp_table[nesqp->hwqp.qp_id-NES_FIRST_QPN] = NULL; kfree(nesqp->allocated_buffer); } @@ -289,7 +290,6 @@ void nes_rem_ref(struct ib_qp *ibqp) struct nes_qp *nesqp; struct nes_vnic *nesvnic = to_nesvnic(ibqp->device); struct nes_device *nesdev = nesvnic->nesdev; - struct nes_adapter *nesadapter = nesdev->nesadapter; struct nes_hw_cqp_wqe *cqp_wqe; struct nes_cqp_request *cqp_request; u32 opcode; @@ -303,8 +303,6 @@ void nes_rem_ref(struct ib_qp *ibqp) } if (atomic_dec_and_test(&nesqp->refcount)) { - nesadapter->qp_table[nesqp->hwqp.qp_id-NES_FIRST_QPN] = NULL; - /* Destroy the QP */ cqp_request = nes_get_cqp_request(nesdev); if (cqp_request == NULL) { diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index 6aa531d..f658ab7 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -74,36 +74,59 @@ atomic_t cm_nodes_destroyed; atomic_t cm_accel_dropped_pkts; atomic_t cm_resets_recvd; -static inline int mini_cm_accelerated(struct nes_cm_core *, struct nes_cm_node *); +static inline int mini_cm_accelerated(struct nes_cm_core *, + struct nes_cm_node *); static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *, - struct nes_vnic *, struct nes_cm_info *); -static int add_ref_cm_node(struct nes_cm_node *); -static int rem_ref_cm_node(struct nes_cm_core *, struct nes_cm_node *); + struct nes_vnic *, struct nes_cm_info *); static int mini_cm_del_listen(struct nes_cm_core *, struct nes_cm_listener *); -static struct sk_buff *form_cm_frame(struct sk_buff *, struct nes_cm_node *, - void *, u32, void *, u32, u8); -static struct sk_buff *get_free_pkt(struct nes_cm_node *cm_node); - static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *, - struct nes_vnic *, - struct ietf_mpa_frame *, - struct nes_cm_info *); + struct nes_vnic *, u16, void *, struct nes_cm_info *); +static int mini_cm_close(struct nes_cm_core *, struct nes_cm_node *); static int mini_cm_accept(struct nes_cm_core *, struct ietf_mpa_frame *, - struct nes_cm_node *); + struct nes_cm_node *); static int mini_cm_reject(struct nes_cm_core *, struct ietf_mpa_frame *, - struct nes_cm_node *); -static int mini_cm_close(struct nes_cm_core *, struct nes_cm_node *); -static int mini_cm_recv_pkt(struct nes_cm_core *, struct nes_vnic *, - struct sk_buff *); + struct nes_cm_node *); +static void mini_cm_recv_pkt(struct nes_cm_core *, struct nes_vnic *, + struct sk_buff *); static int mini_cm_dealloc_core(struct nes_cm_core *); static int mini_cm_get(struct nes_cm_core *); static int mini_cm_set(struct nes_cm_core *, u32, u32); + +static struct sk_buff *form_cm_frame(struct sk_buff *, struct nes_cm_node *, + void *, u32, void *, u32, u8); +static struct sk_buff *get_free_pkt(struct nes_cm_node *cm_node); +static int add_ref_cm_node(struct nes_cm_node *); +static int rem_ref_cm_node(struct nes_cm_core *, struct nes_cm_node *); + static int nes_cm_disconn_true(struct nes_qp *); static int nes_cm_post_event(struct nes_cm_event *event); static int nes_disconnect(struct nes_qp *nesqp, int abrupt); static void nes_disconnect_worker(struct work_struct *work); -static int send_ack(struct nes_cm_node *cm_node); + +static int send_mpa_request(struct nes_cm_node *, struct sk_buff *); +static int send_syn(struct nes_cm_node *, u32, struct sk_buff *); +static int send_reset(struct nes_cm_node *, struct sk_buff *); +static int send_ack(struct nes_cm_node *cm_node, struct sk_buff *skb); static int send_fin(struct nes_cm_node *cm_node, struct sk_buff *skb); +static void process_packet(struct nes_cm_node *, struct sk_buff *, + struct nes_cm_core *); + +static void active_open_err(struct nes_cm_node *, struct sk_buff *, int); +static void passive_open_err(struct nes_cm_node *, struct sk_buff *, int); +static void cleanup_retrans_entry(struct nes_cm_node *); +static void handle_rcv_mpa(struct nes_cm_node *, struct sk_buff *, + enum nes_cm_event_type); +static void free_retrans_entry(struct nes_cm_node *cm_node); +static int handle_tcp_options(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb, int optionsize, int passive); + +/* CM event handler functions */ +static void cm_event_connected(struct nes_cm_event *); +static void cm_event_connect_error(struct nes_cm_event *); +static void cm_event_reset(struct nes_cm_event *); +static void cm_event_mpa_req(struct nes_cm_event *); + +static void print_core(struct nes_cm_core *core); /* External CM API Interface */ /* instance of function pointers for client API */ @@ -158,11 +181,11 @@ static struct nes_cm_event *create_event(struct nes_cm_node *cm_node, event->cm_info.loc_port = cm_node->loc_port; event->cm_info.cm_id = cm_node->cm_id; - nes_debug(NES_DBG_CM, "Created event=%p, type=%u, dst_addr=%08x[%x]," - " src_addr=%08x[%x]\n", - event, type, - event->cm_info.loc_addr, event->cm_info.loc_port, - event->cm_info.rem_addr, event->cm_info.rem_port); + nes_debug(NES_DBG_CM, "cm_node=%p Created event=%p, type=%u, " + "dst_addr=%08x[%x], src_addr=%08x[%x]\n", + cm_node, event, type, event->cm_info.loc_addr, + event->cm_info.loc_port, event->cm_info.rem_addr, + event->cm_info.rem_port); nes_cm_post_event(event); return event; @@ -172,14 +195,11 @@ static struct nes_cm_event *create_event(struct nes_cm_node *cm_node, /** * send_mpa_request */ -static int send_mpa_request(struct nes_cm_node *cm_node) +static int send_mpa_request(struct nes_cm_node *cm_node, struct sk_buff *skb) { - struct sk_buff *skb; int ret; - - skb = get_free_pkt(cm_node); if (!skb) { - nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); + nes_debug(NES_DBG_CM, "skb set to NULL\n"); return -1; } @@ -188,9 +208,8 @@ static int send_mpa_request(struct nes_cm_node *cm_node) cm_node->mpa_frame_size, SET_ACK); ret = schedule_nes_timer(cm_node, skb, NES_TIMER_TYPE_SEND, 1, 0); - if (ret < 0) { + if (ret < 0) return ret; - } return 0; } @@ -229,46 +248,12 @@ static int parse_mpa(struct nes_cm_node *cm_node, u8 *buffer, u32 len) /** - * handle_exception_pkt - process an exception packet. - * We have been in a TSA state, and we have now received SW - * TCP/IP traffic should be a FIN request or IP pkt with options - */ -static int handle_exception_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb) -{ - int ret = 0; - struct tcphdr *tcph = tcp_hdr(skb); - - /* first check to see if this a FIN pkt */ - if (tcph->fin) { - /* we need to ACK the FIN request */ - send_ack(cm_node); - - /* check which side we are (client/server) and set next state accordingly */ - if (cm_node->tcp_cntxt.client) - cm_node->state = NES_CM_STATE_CLOSING; - else { - /* we are the server side */ - cm_node->state = NES_CM_STATE_CLOSE_WAIT; - /* since this is a self contained CM we don't wait for */ - /* an APP to close us, just send final FIN immediately */ - ret = send_fin(cm_node, NULL); - cm_node->state = NES_CM_STATE_LAST_ACK; - } - } else { - ret = -EINVAL; - } - - return ret; -} - - -/** * form_cm_frame - get a free packet and build empty frame Use * node info to build. */ -static struct sk_buff *form_cm_frame(struct sk_buff *skb, struct nes_cm_node *cm_node, - void *options, u32 optionsize, void *data, - u32 datasize, u8 flags) +static struct sk_buff *form_cm_frame(struct sk_buff *skb, + struct nes_cm_node *cm_node, void *options, u32 optionsize, + void *data, u32 datasize, u8 flags) { struct tcphdr *tcph; struct iphdr *iph; @@ -332,10 +317,12 @@ static struct sk_buff *form_cm_frame(struct sk_buff *skb, struct nes_cm_node *cm cm_node->tcp_cntxt.loc_seq_num++; tcph->syn = 1; } else - cm_node->tcp_cntxt.loc_seq_num += datasize; /* data (no headers) */ + cm_node->tcp_cntxt.loc_seq_num += datasize; - if (flags & SET_FIN) + if (flags & SET_FIN) { + cm_node->tcp_cntxt.loc_seq_num++; tcph->fin = 1; + } if (flags & SET_RST) tcph->rst = 1; @@ -389,7 +376,7 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, int close_when_complete) { unsigned long flags; - struct nes_cm_core *cm_core; + struct nes_cm_core *cm_core = cm_node->cm_core; struct nes_timer_entry *new_send; int ret = 0; u32 was_timer_set; @@ -411,7 +398,7 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, new_send->close_when_complete = close_when_complete; if (type == NES_TIMER_TYPE_CLOSE) { - new_send->timetosend += (HZ/2); /* TODO: decide on the correct value here */ + new_send->timetosend += (HZ/10); spin_lock_irqsave(&cm_node->recv_list_lock, flags); list_add_tail(&new_send->list, &cm_node->recv_list); spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); @@ -420,36 +407,28 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, if (type == NES_TIMER_TYPE_SEND) { new_send->seq_num = ntohl(tcp_hdr(skb)->seq); atomic_inc(&new_send->skb->users); + spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + cm_node->send_entry = new_send; + add_ref_cm_node(cm_node); + spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); + new_send->timetosend = jiffies + NES_RETRY_TIMEOUT; ret = nes_nic_cm_xmit(new_send->skb, cm_node->netdev); if (ret != NETDEV_TX_OK) { - nes_debug(NES_DBG_CM, "Error sending packet %p (jiffies = %lu)\n", - new_send, jiffies); + nes_debug(NES_DBG_CM, "Error sending packet %p " + "(jiffies = %lu)\n", new_send, jiffies); atomic_dec(&new_send->skb->users); new_send->timetosend = jiffies; } else { cm_packets_sent++; if (!send_retrans) { + cleanup_retrans_entry(cm_node); if (close_when_complete) - rem_ref_cm_node(cm_node->cm_core, cm_node); - dev_kfree_skb_any(new_send->skb); - kfree(new_send); + rem_ref_cm_node(cm_core, cm_node); return ret; } - new_send->timetosend = jiffies + NES_RETRY_TIMEOUT; } - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - list_add_tail(&new_send->list, &cm_node->retrans_list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - } - if (type == NES_TIMER_TYPE_RECV) { - new_send->seq_num = ntohl(tcp_hdr(skb)->seq); - new_send->timetosend = jiffies; - spin_lock_irqsave(&cm_node->recv_list_lock, flags); - list_add_tail(&new_send->list, &cm_node->recv_list); - spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); } - cm_core = cm_node->cm_core; was_timer_set = timer_pending(&cm_core->tcp_timer); @@ -476,23 +455,27 @@ static void nes_cm_timer_tick(unsigned long pass) struct list_head *list_node, *list_node_temp; struct nes_cm_core *cm_core = g_cm_core; struct nes_qp *nesqp; - struct sk_buff *skb; u32 settimer = 0; int ret = NETDEV_TX_OK; - int node_done; + enum nes_cm_node_state last_state; spin_lock_irqsave(&cm_core->ht_lock, flags); - list_for_each_safe(list_node, list_core_temp, &cm_core->connected_nodes) { + list_for_each_safe(list_node, list_core_temp, + &cm_core->connected_nodes) { cm_node = container_of(list_node, struct nes_cm_node, list); add_ref_cm_node(cm_node); spin_unlock_irqrestore(&cm_core->ht_lock, flags); spin_lock_irqsave(&cm_node->recv_list_lock, flags); - list_for_each_safe(list_core, list_node_temp, &cm_node->recv_list) { - recv_entry = container_of(list_core, struct nes_timer_entry, list); - if ((time_after(recv_entry->timetosend, jiffies)) && - (recv_entry->type == NES_TIMER_TYPE_CLOSE)) { - if (nexttimeout > recv_entry->timetosend || !settimer) { + list_for_each_safe(list_core, list_node_temp, + &cm_node->recv_list) { + recv_entry = container_of(list_core, + struct nes_timer_entry, list); + if (!recv_entry) + break; + if (time_after(recv_entry->timetosend, jiffies)) { + if (nexttimeout > recv_entry->timetosend || + !settimer) { nexttimeout = recv_entry->timetosend; settimer = 1; } @@ -501,157 +484,143 @@ static void nes_cm_timer_tick(unsigned long pass) list_del(&recv_entry->list); cm_id = cm_node->cm_id; spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); - if (recv_entry->type == NES_TIMER_TYPE_CLOSE) { - nesqp = (struct nes_qp *)recv_entry->skb; - spin_lock_irqsave(&nesqp->lock, qplockflags); - if (nesqp->cm_id) { - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, refcount = %d: " - "****** HIT A NES_TIMER_TYPE_CLOSE" - " with something to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id, - atomic_read(&nesqp->refcount)); - nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; - nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; - nesqp->ibqp_state = IB_QPS_ERR; - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_cm_disconn(nesqp); - } else { - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, refcount = %d:" - " ****** HIT A NES_TIMER_TYPE_CLOSE" - " with nothing to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id, - atomic_read(&nesqp->refcount)); - nes_rem_ref(&nesqp->ibqp); - } - if (cm_id) - cm_id->rem_ref(cm_id); + nesqp = (struct nes_qp *)recv_entry->skb; + spin_lock_irqsave(&nesqp->lock, qplockflags); + if (nesqp->cm_id) { + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, " + "refcount = %d: HIT A " + "NES_TIMER_TYPE_CLOSE with something " + "to do!!!\n", nesqp->hwqp.qp_id, cm_id, + atomic_read(&nesqp->refcount)); + nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; + nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; + nesqp->ibqp_state = IB_QPS_ERR; + spin_unlock_irqrestore(&nesqp->lock, + qplockflags); + nes_cm_disconn(nesqp); + } else { + spin_unlock_irqrestore(&nesqp->lock, + qplockflags); + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p, " + "refcount = %d: HIT A " + "NES_TIMER_TYPE_CLOSE with nothing " + "to do!!!\n", nesqp->hwqp.qp_id, cm_id, + atomic_read(&nesqp->refcount)); } + if (cm_id) + cm_id->rem_ref(cm_id); + kfree(recv_entry); spin_lock_irqsave(&cm_node->recv_list_lock, flags); } spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - node_done = 0; - list_for_each_safe(list_core, list_node_temp, &cm_node->retrans_list) { - if (node_done) { - break; - } - send_entry = container_of(list_core, struct nes_timer_entry, list); + do { + send_entry = cm_node->send_entry; + if (!send_entry) + continue; if (time_after(send_entry->timetosend, jiffies)) { if (cm_node->state != NES_CM_STATE_TSA) { - if ((nexttimeout > send_entry->timetosend) || !settimer) { - nexttimeout = send_entry->timetosend; + if ((nexttimeout > + send_entry->timetosend) || + !settimer) { + nexttimeout = + send_entry->timetosend; settimer = 1; + continue; } - node_done = 1; - continue; } else { - list_del(&send_entry->list); - skb = send_entry->skb; - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - dev_kfree_skb_any(skb); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + free_retrans_entry(cm_node); continue; } } - if (send_entry->type == NES_TIMER_NODE_CLEANUP) { - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - continue; - } - if ((send_entry->seq_num < cm_node->tcp_cntxt.rem_ack_num) || - (cm_node->state == NES_CM_STATE_TSA) || - (cm_node->state == NES_CM_STATE_CLOSED)) { - skb = send_entry->skb; - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - kfree(send_entry); - dev_kfree_skb_any(skb); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + + if ((cm_node->state == NES_CM_STATE_TSA) || + (cm_node->state == NES_CM_STATE_CLOSED)) { + free_retrans_entry(cm_node); continue; } - if (!send_entry->retranscount || !send_entry->retrycount) { + if (!send_entry->retranscount || + !send_entry->retrycount) { cm_packets_dropped++; - skb = send_entry->skb; - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - dev_kfree_skb_any(skb); - kfree(send_entry); - if (cm_node->state == NES_CM_STATE_SYN_RCVD) { - /* this node never even generated an indication up to the cm */ + last_state = cm_node->state; + cm_node->state = NES_CM_STATE_CLOSED; + free_retrans_entry(cm_node); + spin_unlock_irqrestore( + &cm_node->retrans_list_lock, flags); + if (last_state == NES_CM_STATE_SYN_RCVD) rem_ref_cm_node(cm_core, cm_node); - } else { - cm_node->state = NES_CM_STATE_CLOSED; - create_event(cm_node, NES_CM_EVENT_ABORTED); - } - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + else + create_event(cm_node, + NES_CM_EVENT_ABORTED); + spin_lock_irqsave(&cm_node->retrans_list_lock, + flags); continue; } - /* this seems like the correct place, but leave send entry unprotected */ - /* spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); */ atomic_inc(&send_entry->skb->users); cm_packets_retrans++; - nes_debug(NES_DBG_CM, "Retransmitting send_entry %p for node %p," - " jiffies = %lu, time to send = %lu, retranscount = %u, " - "send_entry->seq_num = 0x%08X, cm_node->tcp_cntxt.rem_ack_num = 0x%08X\n", - send_entry, cm_node, jiffies, send_entry->timetosend, send_entry->retranscount, - send_entry->seq_num, cm_node->tcp_cntxt.rem_ack_num); - - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); + nes_debug(NES_DBG_CM, "Retransmitting send_entry %p " + "for node %p, jiffies = %lu, time to send = " + "%lu, retranscount = %u, send_entry->seq_num = " + "0x%08X, cm_node->tcp_cntxt.rem_ack_num = " + "0x%08X\n", send_entry, cm_node, jiffies, + send_entry->timetosend, + send_entry->retranscount, + send_entry->seq_num, + cm_node->tcp_cntxt.rem_ack_num); + + spin_unlock_irqrestore(&cm_node->retrans_list_lock, + flags); ret = nes_nic_cm_xmit(send_entry->skb, cm_node->netdev); + spin_lock_irqsave(&cm_node->retrans_list_lock, flags); if (ret != NETDEV_TX_OK) { + nes_debug(NES_DBG_CM, "rexmit failed for " + "node=%p\n", cm_node); cm_packets_bounced++; atomic_dec(&send_entry->skb->users); send_entry->retrycount--; nexttimeout = jiffies + NES_SHORT_TIME; settimer = 1; - node_done = 1; - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); continue; } else { cm_packets_sent++; } - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - list_del(&send_entry->list); - nes_debug(NES_DBG_CM, "Packet Sent: retrans count = %u, retry count = %u.\n", - send_entry->retranscount, send_entry->retrycount); + nes_debug(NES_DBG_CM, "Packet Sent: retrans count = " + "%u, retry count = %u.\n", + send_entry->retranscount, + send_entry->retrycount); if (send_entry->send_retrans) { send_entry->retranscount--; - send_entry->timetosend = jiffies + NES_RETRY_TIMEOUT; - if (nexttimeout > send_entry->timetosend || !settimer) { + send_entry->timetosend = jiffies + + NES_RETRY_TIMEOUT; + if (nexttimeout > send_entry->timetosend || + !settimer) { nexttimeout = send_entry->timetosend; settimer = 1; } - list_add(&send_entry->list, &cm_node->retrans_list); - continue; } else { int close_when_complete; - skb = send_entry->skb; - close_when_complete = send_entry->close_when_complete; - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - if (close_when_complete) { - BUG_ON(atomic_read(&cm_node->ref_count) == 1); - rem_ref_cm_node(cm_core, cm_node); - } - dev_kfree_skb_any(skb); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - continue; + close_when_complete = + send_entry->close_when_complete; + nes_debug(NES_DBG_CM, "cm_node=%p state=%d\n", + cm_node, cm_node->state); + free_retrans_entry(cm_node); + if (close_when_complete) + rem_ref_cm_node(cm_node->cm_core, + cm_node); } - } - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - - rem_ref_cm_node(cm_core, cm_node); + } while (0); + spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); + rem_ref_cm_node(cm_node->cm_core, cm_node); spin_lock_irqsave(&cm_core->ht_lock, flags); - if (ret != NETDEV_TX_OK) + if (ret != NETDEV_TX_OK) { + nes_debug(NES_DBG_CM, "rexmit failed for cm_node=%p\n", + cm_node); break; + } } spin_unlock_irqrestore(&cm_core->ht_lock, flags); @@ -667,14 +636,14 @@ static void nes_cm_timer_tick(unsigned long pass) /** * send_syn */ -static int send_syn(struct nes_cm_node *cm_node, u32 sendack) +static int send_syn(struct nes_cm_node *cm_node, u32 sendack, + struct sk_buff *skb) { int ret; int flags = SET_SYN; - struct sk_buff *skb; char optionsbuffer[sizeof(struct option_mss) + - sizeof(struct option_windowscale) + - sizeof(struct option_base) + 1]; + sizeof(struct option_windowscale) + sizeof(struct option_base) + + TCP_OPTIONS_PADDING]; int optionssize = 0; /* Sending MSS option */ @@ -695,8 +664,7 @@ static int send_syn(struct nes_cm_node *cm_node, u32 sendack) options->as_windowscale.shiftcount = cm_node->tcp_cntxt.rcv_wscale; optionssize += sizeof(struct option_windowscale); - if (sendack && !(NES_DRV_OPT_SUPRESS_OPTION_BC & nes_drv_opt) - ) { + if (sendack && !(NES_DRV_OPT_SUPRESS_OPTION_BC & nes_drv_opt)) { options = (union all_known_options *)&optionsbuffer[optionssize]; options->as_base.optionnum = OPTION_NUMBER_WRITE0; options->as_base.length = sizeof(struct option_base); @@ -714,7 +682,8 @@ static int send_syn(struct nes_cm_node *cm_node, u32 sendack) options->as_end = OPTION_NUMBER_END; optionssize += 1; - skb = get_free_pkt(cm_node); + if (!skb) + skb = get_free_pkt(cm_node); if (!skb) { nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); return -1; @@ -733,18 +702,18 @@ static int send_syn(struct nes_cm_node *cm_node, u32 sendack) /** * send_reset */ -static int send_reset(struct nes_cm_node *cm_node) +static int send_reset(struct nes_cm_node *cm_node, struct sk_buff *skb) { int ret; - struct sk_buff *skb = get_free_pkt(cm_node); int flags = SET_RST | SET_ACK; + if (!skb) + skb = get_free_pkt(cm_node); if (!skb) { nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); return -1; } - add_ref_cm_node(cm_node); form_cm_frame(skb, cm_node, NULL, 0, NULL, 0, flags); ret = schedule_nes_timer(cm_node, skb, NES_TIMER_TYPE_SEND, 0, 1); @@ -755,10 +724,12 @@ static int send_reset(struct nes_cm_node *cm_node) /** * send_ack */ -static int send_ack(struct nes_cm_node *cm_node) +static int send_ack(struct nes_cm_node *cm_node, struct sk_buff *skb) { int ret; - struct sk_buff *skb = get_free_pkt(cm_node); + + if (!skb) + skb = get_free_pkt(cm_node); if (!skb) { nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); @@ -922,7 +893,8 @@ static int add_hte_node(struct nes_cm_core *cm_core, struct nes_cm_node *cm_node if (!cm_node || !cm_core) return -EINVAL; - nes_debug(NES_DBG_CM, "Adding Node to Active Connection HT\n"); + nes_debug(NES_DBG_CM, "Adding Node %p to Active Connection HT\n", + cm_node); /* first, make an index into our hash table */ hashkey = make_hashkey(cm_node->loc_port, cm_node->loc_addr, @@ -946,10 +918,35 @@ static int add_hte_node(struct nes_cm_core *cm_core, struct nes_cm_node *cm_node * mini_cm_dec_refcnt_listen */ static int mini_cm_dec_refcnt_listen(struct nes_cm_core *cm_core, - struct nes_cm_listener *listener, int free_hanging_nodes) + struct nes_cm_listener *listener, int free_hanging_nodes) { int ret = 1; unsigned long flags; + struct list_head *list_pos = NULL; + struct list_head *list_temp = NULL; + struct nes_cm_node *cm_node = NULL; + + nes_debug(NES_DBG_CM, "attempting listener= %p free_nodes= %d, " + "refcnt=%d\n", listener, free_hanging_nodes, + atomic_read(&listener->ref_count)); + /* free non-accelerated child nodes for this listener */ + if (free_hanging_nodes) { + spin_lock_irqsave(&cm_core->ht_lock, flags); + list_for_each_safe(list_pos, list_temp, + &g_cm_core->connected_nodes) { + cm_node = container_of(list_pos, struct nes_cm_node, + list); + if ((cm_node->listener == listener) && + (!cm_node->accelerated)) { + cleanup_retrans_entry(cm_node); + spin_unlock_irqrestore(&cm_core->ht_lock, + flags); + send_reset(cm_node, NULL); + spin_lock_irqsave(&cm_core->ht_lock, flags); + } + } + spin_unlock_irqrestore(&cm_core->ht_lock, flags); + } spin_lock_irqsave(&cm_core->listen_list_lock, flags); if (!atomic_dec_return(&listener->ref_count)) { list_del(&listener->list); @@ -1067,18 +1064,18 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, cm_node->loc_port = cm_info->loc_port; cm_node->rem_port = cm_info->rem_port; cm_node->send_write0 = send_first; - nes_debug(NES_DBG_CM, "Make node addresses : loc = " NIPQUAD_FMT ":%x, rem = " NIPQUAD_FMT ":%x\n", - HIPQUAD(cm_node->loc_addr), cm_node->loc_port, - HIPQUAD(cm_node->rem_addr), cm_node->rem_port); + nes_debug(NES_DBG_CM, "Make node addresses : loc = " NIPQUAD_FMT + ":%x, rem = " NIPQUAD_FMT ":%x\n", + HIPQUAD(cm_node->loc_addr), cm_node->loc_port, + HIPQUAD(cm_node->rem_addr), cm_node->rem_port); cm_node->listener = listener; cm_node->netdev = nesvnic->netdev; cm_node->cm_id = cm_info->cm_id; memcpy(cm_node->loc_mac, nesvnic->netdev->dev_addr, ETH_ALEN); - nes_debug(NES_DBG_CM, "listener=%p, cm_id=%p\n", - cm_node->listener, cm_node->cm_id); + nes_debug(NES_DBG_CM, "listener=%p, cm_id=%p\n", cm_node->listener, + cm_node->cm_id); - INIT_LIST_HEAD(&cm_node->retrans_list); spin_lock_init(&cm_node->retrans_list_lock); INIT_LIST_HEAD(&cm_node->recv_list); spin_lock_init(&cm_node->recv_list_lock); @@ -1142,10 +1139,9 @@ static int add_ref_cm_node(struct nes_cm_node *cm_node) * rem_ref_cm_node - destroy an instance of a cm node */ static int rem_ref_cm_node(struct nes_cm_core *cm_core, - struct nes_cm_node *cm_node) + struct nes_cm_node *cm_node) { unsigned long flags, qplockflags; - struct nes_timer_entry *send_entry; struct nes_timer_entry *recv_entry; struct iw_cm_id *cm_id; struct list_head *list_core, *list_node_temp; @@ -1169,48 +1165,33 @@ static int rem_ref_cm_node(struct nes_cm_core *cm_core, atomic_dec(&cm_node->listener->pend_accepts_cnt); BUG_ON(atomic_read(&cm_node->listener->pend_accepts_cnt) < 0); } - - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - list_for_each_safe(list_core, list_node_temp, &cm_node->retrans_list) { - send_entry = container_of(list_core, struct nes_timer_entry, list); - list_del(&send_entry->list); - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - dev_kfree_skb_any(send_entry->skb); - kfree(send_entry); - spin_lock_irqsave(&cm_node->retrans_list_lock, flags); - continue; - } - spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); - + BUG_ON(cm_node->send_entry); spin_lock_irqsave(&cm_node->recv_list_lock, flags); list_for_each_safe(list_core, list_node_temp, &cm_node->recv_list) { - recv_entry = container_of(list_core, struct nes_timer_entry, list); + recv_entry = container_of(list_core, struct nes_timer_entry, + list); list_del(&recv_entry->list); cm_id = cm_node->cm_id; spin_unlock_irqrestore(&cm_node->recv_list_lock, flags); - if (recv_entry->type == NES_TIMER_TYPE_CLOSE) { - nesqp = (struct nes_qp *)recv_entry->skb; - spin_lock_irqsave(&nesqp->lock, qplockflags); - if (nesqp->cm_id) { - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: ****** HIT A NES_TIMER_TYPE_CLOSE" - " with something to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id); - nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; - nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; - nesqp->ibqp_state = IB_QPS_ERR; - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_cm_disconn(nesqp); - } else { - spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: ****** HIT A NES_TIMER_TYPE_CLOSE" - " with nothing to do!!! ******\n", - nesqp->hwqp.qp_id, cm_id); - nes_rem_ref(&nesqp->ibqp); - } - cm_id->rem_ref(cm_id); - } else if (recv_entry->type == NES_TIMER_TYPE_RECV) { - dev_kfree_skb_any(recv_entry->skb); + nesqp = (struct nes_qp *)recv_entry->skb; + spin_lock_irqsave(&nesqp->lock, qplockflags); + if (nesqp->cm_id) { + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: HIT A " + "NES_TIMER_TYPE_CLOSE with something to do!\n", + nesqp->hwqp.qp_id, cm_id); + nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; + nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; + nesqp->ibqp_state = IB_QPS_ERR; + spin_unlock_irqrestore(&nesqp->lock, qplockflags); + nes_cm_disconn(nesqp); + } else { + spin_unlock_irqrestore(&nesqp->lock, qplockflags); + nes_debug(NES_DBG_CM, "QP%u: cm_id = %p: HIT A " + "NES_TIMER_TYPE_CLOSE with nothing to do!\n", + nesqp->hwqp.qp_id, cm_id); } + cm_id->rem_ref(cm_id); + kfree(recv_entry); spin_lock_irqsave(&cm_node->recv_list_lock, flags); } @@ -1221,23 +1202,31 @@ static int rem_ref_cm_node(struct nes_cm_core *cm_core, } else { if (cm_node->apbvt_set && cm_node->nesvnic) { nes_manage_apbvt(cm_node->nesvnic, cm_node->loc_port, - PCI_FUNC(cm_node->nesvnic->nesdev->pcidev->devfn), - NES_MANAGE_APBVT_DEL); + PCI_FUNC( + cm_node->nesvnic->nesdev->pcidev->devfn), + NES_MANAGE_APBVT_DEL); } } - kfree(cm_node); atomic_dec(&cm_core->node_cnt); atomic_inc(&cm_nodes_destroyed); + nesqp = cm_node->nesqp; + if (nesqp) { + nesqp->cm_node = NULL; + nes_rem_ref(&nesqp->ibqp); + cm_node->nesqp = NULL; + } + cm_node->freed = 1; + kfree(cm_node); return 0; } - /** * process_options */ -static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 optionsize, u32 syn_packet) +static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, + u32 optionsize, u32 syn_packet) { u32 tmp; u32 offset = 0; @@ -1247,35 +1236,37 @@ static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 opti while (offset < optionsize) { all_options = (union all_known_options *)(optionsloc + offset); switch (all_options->as_base.optionnum) { - case OPTION_NUMBER_END: - offset = optionsize; - break; - case OPTION_NUMBER_NONE: - offset += 1; - continue; - case OPTION_NUMBER_MSS: - nes_debug(NES_DBG_CM, "%s: MSS Length: %d Offset: %d Size: %d\n", - __func__, - all_options->as_mss.length, offset, optionsize); - got_mss_option = 1; - if (all_options->as_mss.length != 4) { - return 1; - } else { - tmp = ntohs(all_options->as_mss.mss); - if (tmp > 0 && tmp < cm_node->tcp_cntxt.mss) - cm_node->tcp_cntxt.mss = tmp; - } - break; - case OPTION_NUMBER_WINDOW_SCALE: - cm_node->tcp_cntxt.snd_wscale = all_options->as_windowscale.shiftcount; - break; - case OPTION_NUMBER_WRITE0: - cm_node->send_write0 = 1; - break; - default: - nes_debug(NES_DBG_CM, "TCP Option not understood: %x\n", - all_options->as_base.optionnum); - break; + case OPTION_NUMBER_END: + offset = optionsize; + break; + case OPTION_NUMBER_NONE: + offset += 1; + continue; + case OPTION_NUMBER_MSS: + nes_debug(NES_DBG_CM, "%s: MSS Length: %d Offset: %d " + "Size: %d\n", __func__, + all_options->as_mss.length, offset, optionsize); + got_mss_option = 1; + if (all_options->as_mss.length != 4) { + return 1; + } else { + tmp = ntohs(all_options->as_mss.mss); + if (tmp > 0 && tmp < + cm_node->tcp_cntxt.mss) + cm_node->tcp_cntxt.mss = tmp; + } + break; + case OPTION_NUMBER_WINDOW_SCALE: + cm_node->tcp_cntxt.snd_wscale = + all_options->as_windowscale.shiftcount; + break; + case OPTION_NUMBER_WRITE0: + cm_node->send_write0 = 1; + break; + default: + nes_debug(NES_DBG_CM, "TCP Option not understood: %x\n", + all_options->as_base.optionnum); + break; } offset += all_options->as_base.length; } @@ -1284,300 +1275,491 @@ static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 opti return 0; } +inline void drop_packet(struct sk_buff *skb) +{ + atomic_inc(&cm_accel_dropped_pkts); + dev_kfree_skb_any(skb); +} -/** - * process_packet - */ -static int process_packet(struct nes_cm_node *cm_node, struct sk_buff *skb, - struct nes_cm_core *cm_core) +static void handle_fin_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) { - int optionsize; - int datasize; - int ret = 0; - struct tcphdr *tcph = tcp_hdr(skb); - u32 inc_sequence; - if (cm_node->state == NES_CM_STATE_SYN_SENT && tcph->syn) { - inc_sequence = ntohl(tcph->seq); - cm_node->tcp_cntxt.rcv_nxt = inc_sequence; + atomic_inc(&cm_resets_recvd); + nes_debug(NES_DBG_CM, "Received FIN, cm_node = %p, state = %u. " + "refcnt=%d\n", cm_node, cm_node->state, + atomic_read(&cm_node->ref_count)); + cm_node->tcp_cntxt.rcv_nxt++; + cleanup_retrans_entry(cm_node); + switch (cm_node->state) { + case NES_CM_STATE_SYN_RCVD: + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_MPAREQ_SENT: + cm_node->state = NES_CM_STATE_LAST_ACK; + send_fin(cm_node, skb); + break; + case NES_CM_STATE_FIN_WAIT1: + cm_node->state = NES_CM_STATE_CLOSING; + send_ack(cm_node, skb); + break; + case NES_CM_STATE_FIN_WAIT2: + cm_node->state = NES_CM_STATE_TIME_WAIT; + send_ack(cm_node, skb); + cm_node->state = NES_CM_STATE_CLOSED; + break; + case NES_CM_STATE_TSA: + default: + nes_debug(NES_DBG_CM, "Error Rcvd FIN for node-%p state = %d\n", + cm_node, cm_node->state); + drop_packet(skb); + break; } +} - if ((!tcph) || (cm_node->state == NES_CM_STATE_TSA)) { - BUG_ON(!tcph); - atomic_inc(&cm_accel_dropped_pkts); - return -1; - } - if (tcph->rst) { - atomic_inc(&cm_resets_recvd); - nes_debug(NES_DBG_CM, "Received Reset, cm_node = %p, state = %u. refcnt=%d\n", - cm_node, cm_node->state, atomic_read(&cm_node->ref_count)); - switch (cm_node->state) { - case NES_CM_STATE_LISTENING: - rem_ref_cm_node(cm_core, cm_node); - break; - case NES_CM_STATE_TSA: - case NES_CM_STATE_CLOSED: - break; - case NES_CM_STATE_SYN_RCVD: - nes_debug(NES_DBG_CM, "Received a reset for local 0x%08X:%04X," - " remote 0x%08X:%04X, node state = %u\n", - cm_node->loc_addr, cm_node->loc_port, - cm_node->rem_addr, cm_node->rem_port, - cm_node->state); - rem_ref_cm_node(cm_core, cm_node); - break; - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_MPAREQ_SENT: - default: - nes_debug(NES_DBG_CM, "Received a reset for local 0x%08X:%04X," - " remote 0x%08X:%04X, node state = %u refcnt=%d\n", - cm_node->loc_addr, cm_node->loc_port, - cm_node->rem_addr, cm_node->rem_port, - cm_node->state, atomic_read(&cm_node->ref_count)); - /* create event */ - cm_node->state = NES_CM_STATE_CLOSED; +static void handle_rst_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ - create_event(cm_node, NES_CM_EVENT_ABORTED); - break; + int reset = 0; /* whether to send reset in case of err.. */ + atomic_inc(&cm_resets_recvd); + nes_debug(NES_DBG_CM, "Received Reset, cm_node = %p, state = %u." + " refcnt=%d\n", cm_node, cm_node->state, + atomic_read(&cm_node->ref_count)); + cleanup_retrans_entry(cm_node); + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_MPAREQ_SENT: + nes_debug(NES_DBG_CM, "%s[%u] create abort for cm_node=%p " + "listener=%p state=%d\n", __func__, __LINE__, cm_node, + cm_node->listener, cm_node->state); + active_open_err(cm_node, skb, reset); + break; + /* For PASSIVE open states, remove the cm_node event */ + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_SYN_RCVD: + case NES_CM_STATE_LISTENING: + nes_debug(NES_DBG_CM, "Bad state %s[%u]\n", __func__, __LINE__); + passive_open_err(cm_node, skb, reset); + break; + case NES_CM_STATE_TSA: + default: + break; + } +} +static void handle_rcv_mpa(struct nes_cm_node *cm_node, struct sk_buff *skb, + enum nes_cm_event_type type) +{ + + int ret; + int datasize = skb->len; + u8 *dataloc = skb->data; + ret = parse_mpa(cm_node, dataloc, datasize); + if (ret < 0) { + nes_debug(NES_DBG_CM, "didn't like MPA Request\n"); + if (type == NES_CM_EVENT_CONNECTED) { + nes_debug(NES_DBG_CM, "%s[%u] create abort for " + "cm_node=%p listener=%p state=%d\n", __func__, + __LINE__, cm_node, cm_node->listener, + cm_node->state); + active_open_err(cm_node, skb, 1); + } else { + passive_open_err(cm_node, skb, 1); } - return -1; + } else { + cleanup_retrans_entry(cm_node); + dev_kfree_skb_any(skb); + if (type == NES_CM_EVENT_CONNECTED) + cm_node->state = NES_CM_STATE_TSA; + create_event(cm_node, type); + + } + return ; +} + +static void indicate_pkt_err(struct nes_cm_node *cm_node, struct sk_buff *skb) +{ + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_MPAREQ_SENT: + nes_debug(NES_DBG_CM, "%s[%u] create abort for cm_node=%p " + "listener=%p state=%d\n", __func__, __LINE__, cm_node, + cm_node->listener, cm_node->state); + active_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_SYN_RCVD: + passive_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_TSA: + default: + drop_packet(skb); } +} + +static int check_syn(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb) +{ + int err; + + err = ((ntohl(tcph->ack_seq) == cm_node->tcp_cntxt.loc_seq_num))? 0 : 1; + if (err) + active_open_err(cm_node, skb, 1); + + return err; +} + +static int check_seq(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb) +{ + int err = 0; + u32 seq; + u32 ack_seq; + u32 loc_seq_num = cm_node->tcp_cntxt.loc_seq_num; + u32 rcv_nxt = cm_node->tcp_cntxt.rcv_nxt; + u32 rcv_wnd; + seq = ntohl(tcph->seq); + ack_seq = ntohl(tcph->ack_seq); + rcv_wnd = cm_node->tcp_cntxt.rcv_wnd; + if (ack_seq != loc_seq_num) + err = 1; + else if ((seq + rcv_wnd) < rcv_nxt) + err = 1; + if (err) { + nes_debug(NES_DBG_CM, "%s[%u] create abort for cm_node=%p " + "listener=%p state=%d\n", __func__, __LINE__, cm_node, + cm_node->listener, cm_node->state); + indicate_pkt_err(cm_node, skb); + nes_debug(NES_DBG_CM, "seq ERROR cm_node =%p seq=0x%08X " + "rcv_nxt=0x%08X rcv_wnd=0x%x\n", cm_node, seq, rcv_nxt, + rcv_wnd); + } + return err; +} + +/* + * handle_syn_pkt() is for Passive node. The syn packet is received when a node + * is created with a listener or it may comein as rexmitted packet which in + * that case will be just dropped. + */ + +static void handle_syn_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ + int ret; + u32 inc_sequence; + int optionsize; optionsize = (tcph->doff << 2) - sizeof(struct tcphdr); + skb_pull(skb, tcph->doff << 2); + inc_sequence = ntohl(tcph->seq); - skb_pull(skb, ip_hdr(skb)->ihl << 2); + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_MPAREQ_SENT: + /* Rcvd syn on active open connection*/ + active_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_LISTENING: + /* Passive OPEN */ + cm_node->accept_pend = 1; + atomic_inc(&cm_node->listener->pend_accepts_cnt); + if (atomic_read(&cm_node->listener->pend_accepts_cnt) > + cm_node->listener->backlog) { + nes_debug(NES_DBG_CM, "drop syn due to backlog " + "pressure \n"); + cm_backlog_drops++; + passive_open_err(cm_node, skb, 0); + break; + } + ret = handle_tcp_options(cm_node, tcph, skb, optionsize, + 1); + if (ret) { + passive_open_err(cm_node, skb, 0); + /* drop pkt */ + break; + } + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + 1; + BUG_ON(cm_node->send_entry); + cm_node->state = NES_CM_STATE_SYN_RCVD; + send_syn(cm_node, 1, skb); + break; + case NES_CM_STATE_TSA: + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_MPAREQ_RCVD: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_CLOSING: + case NES_CM_STATE_UNKNOWN: + case NES_CM_STATE_CLOSED: + default: + drop_packet(skb); + break; + } +} + +static void handle_synack_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ + + int ret; + u32 inc_sequence; + int optionsize; + + optionsize = (tcph->doff << 2) - sizeof(struct tcphdr); skb_pull(skb, tcph->doff << 2); + inc_sequence = ntohl(tcph->seq); + switch (cm_node->state) { + case NES_CM_STATE_SYN_SENT: + /* active open */ + if (check_syn(cm_node, tcph, skb)) + return; + cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); + /* setup options */ + ret = handle_tcp_options(cm_node, tcph, skb, optionsize, 0); + if (ret) { + nes_debug(NES_DBG_CM, "cm_node=%p tcp_options failed\n", + cm_node); + break; + } + cleanup_retrans_entry(cm_node); + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + 1; + send_mpa_request(cm_node, skb); + cm_node->state = NES_CM_STATE_MPAREQ_SENT; + break; + case NES_CM_STATE_MPAREQ_RCVD: + /* passive open, so should not be here */ + passive_open_err(cm_node, skb, 1); + break; + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_TSA: + case NES_CM_STATE_CLOSING: + case NES_CM_STATE_UNKNOWN: + case NES_CM_STATE_CLOSED: + case NES_CM_STATE_MPAREQ_SENT: + default: + drop_packet(skb); + break; + } +} - datasize = skb->len; +static void handle_ack_pkt(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct tcphdr *tcph) +{ + int datasize = 0; + u32 inc_sequence; + u32 rem_seq_ack; + u32 rem_seq; + if (check_seq(cm_node, tcph, skb)) + return; + + skb_pull(skb, tcph->doff << 2); inc_sequence = ntohl(tcph->seq); - nes_debug(NES_DBG_CM, "datasize = %u, sequence = 0x%08X, ack_seq = 0x%08X," - " rcv_nxt = 0x%08X Flags: %s %s.\n", - datasize, inc_sequence, ntohl(tcph->ack_seq), - cm_node->tcp_cntxt.rcv_nxt, (tcph->syn ? "SYN":""), - (tcph->ack ? "ACK":"")); - - if (!tcph->syn && (inc_sequence != cm_node->tcp_cntxt.rcv_nxt) - ) { - nes_debug(NES_DBG_CM, "dropping packet, datasize = %u, sequence = 0x%08X," - " ack_seq = 0x%08X, rcv_nxt = 0x%08X Flags: %s.\n", - datasize, inc_sequence, ntohl(tcph->ack_seq), - cm_node->tcp_cntxt.rcv_nxt, (tcph->ack ? "ACK":"")); - if (cm_node->state == NES_CM_STATE_LISTENING) { - rem_ref_cm_node(cm_core, cm_node); + rem_seq = ntohl(tcph->seq); + rem_seq_ack = ntohl(tcph->ack_seq); + datasize = skb->len; + + switch (cm_node->state) { + case NES_CM_STATE_SYN_RCVD: + /* Passive OPEN */ + cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); + cm_node->state = NES_CM_STATE_ESTABLISHED; + if (datasize) { + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; + cm_node->state = NES_CM_STATE_MPAREQ_RCVD; + handle_rcv_mpa(cm_node, skb, NES_CM_EVENT_MPA_REQ); + } else { /* rcvd ACK only */ + dev_kfree_skb_any(skb); + cleanup_retrans_entry(cm_node); + } + break; + case NES_CM_STATE_ESTABLISHED: + /* Passive OPEN */ + /* We expect mpa frame to be received only */ + if (datasize) { + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; + cm_node->state = NES_CM_STATE_MPAREQ_RCVD; + handle_rcv_mpa(cm_node, skb, + NES_CM_EVENT_MPA_REQ); + } else + drop_packet(skb); + break; + case NES_CM_STATE_MPAREQ_SENT: + cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); + if (datasize) { + cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; + handle_rcv_mpa(cm_node, skb, NES_CM_EVENT_CONNECTED); + } else { /* Could be just an ack pkt.. */ + cleanup_retrans_entry(cm_node); + dev_kfree_skb_any(skb); } - return -1; + break; + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_TSA: + case NES_CM_STATE_CLOSED: + case NES_CM_STATE_MPAREQ_RCVD: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_CLOSING: + case NES_CM_STATE_UNKNOWN: + default: + drop_packet(skb); + break; } +} - cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize; +static int handle_tcp_options(struct nes_cm_node *cm_node, struct tcphdr *tcph, + struct sk_buff *skb, int optionsize, int passive) +{ + u8 *optionsloc = (u8 *)&tcph[1]; if (optionsize) { - u8 *optionsloc = (u8 *)&tcph[1]; - if (process_options(cm_node, optionsloc, optionsize, (u32)tcph->syn)) { - nes_debug(NES_DBG_CM, "%s: Node %p, Sending RESET\n", __func__, cm_node); - send_reset(cm_node); - if (cm_node->state != NES_CM_STATE_SYN_SENT) - rem_ref_cm_node(cm_core, cm_node); - return 0; + if (process_options(cm_node, optionsloc, optionsize, + (u32)tcph->syn)) { + nes_debug(NES_DBG_CM, "%s: Node %p, Sending RESET\n", + __func__, cm_node); + if (passive) + passive_open_err(cm_node, skb, 0); + else + active_open_err(cm_node, skb, 0); + return 1; } - } else if (tcph->syn) - cm_node->tcp_cntxt.mss = NES_CM_DEFAULT_MSS; + } cm_node->tcp_cntxt.snd_wnd = ntohs(tcph->window) << cm_node->tcp_cntxt.snd_wscale; - if (cm_node->tcp_cntxt.snd_wnd > cm_node->tcp_cntxt.max_snd_wnd) { + if (cm_node->tcp_cntxt.snd_wnd > cm_node->tcp_cntxt.max_snd_wnd) cm_node->tcp_cntxt.max_snd_wnd = cm_node->tcp_cntxt.snd_wnd; - } + return 0; +} - if (tcph->ack) { - cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq); - switch (cm_node->state) { - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - /* read and stash current sequence number */ - if (cm_node->tcp_cntxt.rem_ack_num != cm_node->tcp_cntxt.loc_seq_num) { - nes_debug(NES_DBG_CM, "ERROR - cm_node->tcp_cntxt.rem_ack_num !=" - " cm_node->tcp_cntxt.loc_seq_num\n"); - send_reset(cm_node); - return 0; - } - if (cm_node->state == NES_CM_STATE_SYN_SENT) - cm_node->state = NES_CM_STATE_ONE_SIDE_ESTABLISHED; - else { - cm_node->state = NES_CM_STATE_ESTABLISHED; - } - break; - case NES_CM_STATE_LAST_ACK: - cm_node->state = NES_CM_STATE_CLOSED; - break; - case NES_CM_STATE_FIN_WAIT1: - cm_node->state = NES_CM_STATE_FIN_WAIT2; - break; - case NES_CM_STATE_CLOSING: - cm_node->state = NES_CM_STATE_TIME_WAIT; - /* need to schedule this to happen in 2MSL timeouts */ - cm_node->state = NES_CM_STATE_CLOSED; - break; - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_MPAREQ_SENT: - case NES_CM_STATE_CLOSE_WAIT: - case NES_CM_STATE_TIME_WAIT: - case NES_CM_STATE_CLOSED: - break; - case NES_CM_STATE_LISTENING: - nes_debug(NES_DBG_CM, "Received an ACK on a listening port (SYN %d)\n", tcph->syn); - cm_node->tcp_cntxt.loc_seq_num = ntohl(tcph->ack_seq); - send_reset(cm_node); - /* send_reset bumps refcount, this should have been a new node */ - rem_ref_cm_node(cm_core, cm_node); - return -1; - break; - case NES_CM_STATE_TSA: - nes_debug(NES_DBG_CM, "Received a packet with the ack bit set while in TSA state\n"); - break; - case NES_CM_STATE_UNKNOWN: - case NES_CM_STATE_INITED: - case NES_CM_STATE_ACCEPTING: - case NES_CM_STATE_FIN_WAIT2: - default: - nes_debug(NES_DBG_CM, "Received ack from unknown state: %x\n", - cm_node->state); - send_reset(cm_node); - break; - } - } +/* + * active_open_err() will send reset() if flag set.. + * It will also send ABORT event. + */ - if (tcph->syn) { - if (cm_node->state == NES_CM_STATE_LISTENING) { - /* do not exceed backlog */ - atomic_inc(&cm_node->listener->pend_accepts_cnt); - if (atomic_read(&cm_node->listener->pend_accepts_cnt) > - cm_node->listener->backlog) { - nes_debug(NES_DBG_CM, "drop syn due to backlog pressure \n"); - cm_backlog_drops++; - atomic_dec(&cm_node->listener->pend_accepts_cnt); - rem_ref_cm_node(cm_core, cm_node); - return 0; - } - cm_node->accept_pend = 1; +static void active_open_err(struct nes_cm_node *cm_node, struct sk_buff *skb, + int reset) +{ + cleanup_retrans_entry(cm_node); + if (reset) { + nes_debug(NES_DBG_CM, "ERROR active err called for cm_node=%p, " + "state=%d\n", cm_node, cm_node->state); + add_ref_cm_node(cm_node); + send_reset(cm_node, skb); + } else + dev_kfree_skb_any(skb); - } - if (datasize == 0) - cm_node->tcp_cntxt.rcv_nxt ++; + cm_node->state = NES_CM_STATE_CLOSED; + create_event(cm_node, NES_CM_EVENT_ABORTED); +} - if (cm_node->state == NES_CM_STATE_LISTENING) { - cm_node->state = NES_CM_STATE_SYN_RCVD; - send_syn(cm_node, 1); - } - if (cm_node->state == NES_CM_STATE_ONE_SIDE_ESTABLISHED) { - cm_node->state = NES_CM_STATE_ESTABLISHED; - /* send final handshake ACK */ - ret = send_ack(cm_node); - if (ret < 0) - return ret; +/* + * passive_open_err() will either do a reset() or will free up the skb and + * remove the cm_node. + */ - cm_node->state = NES_CM_STATE_MPAREQ_SENT; - ret = send_mpa_request(cm_node); - if (ret < 0) - return ret; - } +static void passive_open_err(struct nes_cm_node *cm_node, struct sk_buff *skb, + int reset) +{ + cleanup_retrans_entry(cm_node); + cm_node->state = NES_CM_STATE_CLOSED; + if (reset) { + nes_debug(NES_DBG_CM, "passive_open_err sending RST for " + "cm_node=%p state =%d\n", cm_node, cm_node->state); + send_reset(cm_node, skb); + } else { + dev_kfree_skb_any(skb); + rem_ref_cm_node(cm_node->cm_core, cm_node); } +} - if (tcph->fin) { - cm_node->tcp_cntxt.rcv_nxt++; - switch (cm_node->state) { - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_ACCEPTING: - case NES_CM_STATE_MPAREQ_SENT: - cm_node->state = NES_CM_STATE_CLOSE_WAIT; - cm_node->state = NES_CM_STATE_LAST_ACK; - ret = send_fin(cm_node, NULL); - break; - case NES_CM_STATE_FIN_WAIT1: - cm_node->state = NES_CM_STATE_CLOSING; - ret = send_ack(cm_node); - break; - case NES_CM_STATE_FIN_WAIT2: - cm_node->state = NES_CM_STATE_TIME_WAIT; - cm_node->tcp_cntxt.loc_seq_num ++; - ret = send_ack(cm_node); - /* need to schedule this to happen in 2MSL timeouts */ - cm_node->state = NES_CM_STATE_CLOSED; - break; - case NES_CM_STATE_CLOSE_WAIT: - case NES_CM_STATE_LAST_ACK: - case NES_CM_STATE_CLOSING: - case NES_CM_STATE_TSA: - default: - nes_debug(NES_DBG_CM, "Received a fin while in %x state\n", - cm_node->state); - ret = -EINVAL; - break; - } +/* + * free_retrans_entry() routines assumes that the retrans_list_lock has + * been acquired before calling. + */ +static void free_retrans_entry(struct nes_cm_node *cm_node) +{ + struct nes_timer_entry *send_entry; + send_entry = cm_node->send_entry; + if (send_entry) { + cm_node->send_entry = NULL; + dev_kfree_skb_any(send_entry->skb); + kfree(send_entry); + rem_ref_cm_node(cm_node->cm_core, cm_node); } +} - if (datasize) { - u8 *dataloc = skb->data; - /* figure out what state we are in and handle transition to next state */ - switch (cm_node->state) { - case NES_CM_STATE_LISTENING: - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - case NES_CM_STATE_FIN_WAIT1: - case NES_CM_STATE_FIN_WAIT2: - case NES_CM_STATE_CLOSE_WAIT: - case NES_CM_STATE_LAST_ACK: - case NES_CM_STATE_CLOSING: - break; - case NES_CM_STATE_MPAREQ_SENT: - /* recv the mpa res frame, ret=frame len (incl priv data) */ - ret = parse_mpa(cm_node, dataloc, datasize); - if (ret < 0) - break; - /* set the req frame payload len in skb */ - /* we are done handling this state, set node to a TSA state */ - cm_node->state = NES_CM_STATE_TSA; - send_ack(cm_node); - create_event(cm_node, NES_CM_EVENT_CONNECTED); - break; - - case NES_CM_STATE_ESTABLISHED: - /* we are expecting an MPA req frame */ - ret = parse_mpa(cm_node, dataloc, datasize); - if (ret < 0) { - break; - } - cm_node->state = NES_CM_STATE_TSA; - send_ack(cm_node); - /* we got a valid MPA request, create an event */ - create_event(cm_node, NES_CM_EVENT_MPA_REQ); - break; - case NES_CM_STATE_TSA: - handle_exception_pkt(cm_node, skb); - break; - case NES_CM_STATE_UNKNOWN: - case NES_CM_STATE_INITED: - default: - ret = -1; - } - } +static void cleanup_retrans_entry(struct nes_cm_node *cm_node) +{ + unsigned long flags; - return ret; + spin_lock_irqsave(&cm_node->retrans_list_lock, flags); + free_retrans_entry(cm_node); + spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags); } +/** + * process_packet + * Returns skb if to be freed, else it will return NULL if already used.. + */ +static void process_packet(struct nes_cm_node *cm_node, struct sk_buff *skb, + struct nes_cm_core *cm_core) +{ + enum nes_tcpip_pkt_type pkt_type = NES_PKT_TYPE_UNKNOWN; + struct tcphdr *tcph = tcp_hdr(skb); + skb_pull(skb, ip_hdr(skb)->ihl << 2); + + nes_debug(NES_DBG_CM, "process_packet: cm_node=%p state =%d syn=%d " + "ack=%d rst=%d fin=%d\n", cm_node, cm_node->state, tcph->syn, + tcph->ack, tcph->rst, tcph->fin); + + if (tcph->rst) + pkt_type = NES_PKT_TYPE_RST; + else if (tcph->syn) { + pkt_type = NES_PKT_TYPE_SYN; + if (tcph->ack) + pkt_type = NES_PKT_TYPE_SYNACK; + } else if (tcph->fin) + pkt_type = NES_PKT_TYPE_FIN; + else if (tcph->ack) + pkt_type = NES_PKT_TYPE_ACK; + + switch (pkt_type) { + case NES_PKT_TYPE_SYN: + handle_syn_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_SYNACK: + handle_synack_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_ACK: + handle_ack_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_RST: + handle_rst_pkt(cm_node, skb, tcph); + break; + case NES_PKT_TYPE_FIN: + handle_fin_pkt(cm_node, skb, tcph); + break; + default: + drop_packet(skb); + break; + } +} /** * mini_cm_listen - create a listen node with params */ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, - struct nes_vnic *nesvnic, struct nes_cm_info *cm_info) + struct nes_vnic *nesvnic, struct nes_cm_info *cm_info) { struct nes_cm_listener *listener; unsigned long flags; @@ -1644,37 +1826,36 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, /** * mini_cm_connect - make a connection node with params */ -static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, - struct nes_vnic *nesvnic, - struct ietf_mpa_frame *mpa_frame, - struct nes_cm_info *cm_info) +struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, + struct nes_vnic *nesvnic, u16 private_data_len, + void *private_data, struct nes_cm_info *cm_info) { int ret = 0; struct nes_cm_node *cm_node; struct nes_cm_listener *loopbackremotelistener; struct nes_cm_node *loopbackremotenode; struct nes_cm_info loopback_cm_info; - - u16 mpa_frame_size = sizeof(struct ietf_mpa_frame) + - ntohs(mpa_frame->priv_data_len); - - cm_info->loc_addr = htonl(cm_info->loc_addr); - cm_info->rem_addr = htonl(cm_info->rem_addr); - cm_info->loc_port = htons(cm_info->loc_port); - cm_info->rem_port = htons(cm_info->rem_port); + u16 mpa_frame_size = sizeof(struct ietf_mpa_frame) + private_data_len; + struct ietf_mpa_frame *mpa_frame = NULL; /* create a CM connection node */ cm_node = make_cm_node(cm_core, nesvnic, cm_info, NULL); if (!cm_node) return NULL; + mpa_frame = &cm_node->mpa_frame; + strcpy(mpa_frame->key, IEFT_MPA_KEY_REQ); + mpa_frame->flags = IETF_MPA_FLAGS_CRC; + mpa_frame->rev = IETF_MPA_VERSION; + mpa_frame->priv_data_len = htons(private_data_len); /* set our node side to client (active) side */ cm_node->tcp_cntxt.client = 1; cm_node->tcp_cntxt.rcv_wscale = NES_CM_DEFAULT_RCV_WND_SCALE; if (cm_info->loc_addr == cm_info->rem_addr) { - loopbackremotelistener = find_listener(cm_core, cm_node->rem_addr, - cm_node->rem_port, NES_CM_LISTENER_ACTIVE_STATE); + loopbackremotelistener = find_listener(cm_core, + ntohl(nesvnic->local_ipaddr), cm_node->rem_port, + NES_CM_LISTENER_ACTIVE_STATE); if (loopbackremotelistener == NULL) { create_event(cm_node, NES_CM_EVENT_ABORTED); } else { @@ -1683,26 +1864,35 @@ static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, loopback_cm_info.loc_port = cm_info->rem_port; loopback_cm_info.rem_port = cm_info->loc_port; loopback_cm_info.cm_id = loopbackremotelistener->cm_id; - loopbackremotenode = make_cm_node(cm_core, nesvnic, &loopback_cm_info, - loopbackremotelistener); + loopbackremotenode = make_cm_node(cm_core, nesvnic, + &loopback_cm_info, loopbackremotelistener); loopbackremotenode->loopbackpartner = cm_node; - loopbackremotenode->tcp_cntxt.rcv_wscale = NES_CM_DEFAULT_RCV_WND_SCALE; + loopbackremotenode->tcp_cntxt.rcv_wscale = + NES_CM_DEFAULT_RCV_WND_SCALE; cm_node->loopbackpartner = loopbackremotenode; - memcpy(loopbackremotenode->mpa_frame_buf, &mpa_frame->priv_data, - mpa_frame_size); - loopbackremotenode->mpa_frame_size = mpa_frame_size - - sizeof(struct ietf_mpa_frame); + memcpy(loopbackremotenode->mpa_frame_buf, private_data, + private_data_len); + loopbackremotenode->mpa_frame_size = private_data_len; - /* we are done handling this state, set node to a TSA state */ + /* we are done handling this state. */ + /* set node to a TSA state */ cm_node->state = NES_CM_STATE_TSA; - cm_node->tcp_cntxt.rcv_nxt = loopbackremotenode->tcp_cntxt.loc_seq_num; - loopbackremotenode->tcp_cntxt.rcv_nxt = cm_node->tcp_cntxt.loc_seq_num; - cm_node->tcp_cntxt.max_snd_wnd = loopbackremotenode->tcp_cntxt.rcv_wnd; - loopbackremotenode->tcp_cntxt.max_snd_wnd = cm_node->tcp_cntxt.rcv_wnd; - cm_node->tcp_cntxt.snd_wnd = loopbackremotenode->tcp_cntxt.rcv_wnd; - loopbackremotenode->tcp_cntxt.snd_wnd = cm_node->tcp_cntxt.rcv_wnd; - cm_node->tcp_cntxt.snd_wscale = loopbackremotenode->tcp_cntxt.rcv_wscale; - loopbackremotenode->tcp_cntxt.snd_wscale = cm_node->tcp_cntxt.rcv_wscale; + cm_node->tcp_cntxt.rcv_nxt = + loopbackremotenode->tcp_cntxt.loc_seq_num; + loopbackremotenode->tcp_cntxt.rcv_nxt = + cm_node->tcp_cntxt.loc_seq_num; + cm_node->tcp_cntxt.max_snd_wnd = + loopbackremotenode->tcp_cntxt.rcv_wnd; + loopbackremotenode->tcp_cntxt.max_snd_wnd = + cm_node->tcp_cntxt.rcv_wnd; + cm_node->tcp_cntxt.snd_wnd = + loopbackremotenode->tcp_cntxt.rcv_wnd; + loopbackremotenode->tcp_cntxt.snd_wnd = + cm_node->tcp_cntxt.rcv_wnd; + cm_node->tcp_cntxt.snd_wscale = + loopbackremotenode->tcp_cntxt.rcv_wscale; + loopbackremotenode->tcp_cntxt.snd_wscale = + cm_node->tcp_cntxt.rcv_wscale; create_event(loopbackremotenode, NES_CM_EVENT_MPA_REQ); } @@ -1712,16 +1902,29 @@ static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, /* set our node side to client (active) side */ cm_node->tcp_cntxt.client = 1; /* init our MPA frame ptr */ - memcpy(&cm_node->mpa_frame, mpa_frame, mpa_frame_size); + memcpy(mpa_frame->priv_data, private_data, private_data_len); + cm_node->mpa_frame_size = mpa_frame_size; /* send a syn and goto syn sent state */ cm_node->state = NES_CM_STATE_SYN_SENT; - ret = send_syn(cm_node, 0); + ret = send_syn(cm_node, 0, NULL); + + if (ret) { + /* error in sending the syn free up the cm_node struct */ + nes_debug(NES_DBG_CM, "Api - connect() FAILED: dest " + "addr=0x%08X, port=0x%04x, cm_node=%p, cm_id = %p.\n", + cm_node->rem_addr, cm_node->rem_port, cm_node, + cm_node->cm_id); + rem_ref_cm_node(cm_node->cm_core, cm_node); + cm_node = NULL; + } - nes_debug(NES_DBG_CM, "Api - connect(): dest addr=0x%08X, port=0x%04x," - " cm_node=%p, cm_id = %p.\n", - cm_node->rem_addr, cm_node->rem_port, cm_node, cm_node->cm_id); + if (cm_node) + nes_debug(NES_DBG_CM, "Api - connect(): dest addr=0x%08X," + "port=0x%04x, cm_node=%p, cm_id = %p.\n", + cm_node->rem_addr, cm_node->rem_port, cm_node, + cm_node->cm_id); return cm_node; } @@ -1731,8 +1934,8 @@ static struct nes_cm_node *mini_cm_connect(struct nes_cm_core *cm_core, * mini_cm_accept - accept a connection * This function is never called */ -static int mini_cm_accept(struct nes_cm_core *cm_core, struct ietf_mpa_frame *mpa_frame, - struct nes_cm_node *cm_node) +static int mini_cm_accept(struct nes_cm_core *cm_core, + struct ietf_mpa_frame *mpa_frame, struct nes_cm_node *cm_node) { return 0; } @@ -1742,32 +1945,26 @@ static int mini_cm_accept(struct nes_cm_core *cm_core, struct ietf_mpa_frame *mp * mini_cm_reject - reject and teardown a connection */ static int mini_cm_reject(struct nes_cm_core *cm_core, - struct ietf_mpa_frame *mpa_frame, - struct nes_cm_node *cm_node) + struct ietf_mpa_frame *mpa_frame, struct nes_cm_node *cm_node) { int ret = 0; - struct sk_buff *skb; - u16 mpa_frame_size = sizeof(struct ietf_mpa_frame) + - ntohs(mpa_frame->priv_data_len); - skb = get_free_pkt(cm_node); - if (!skb) { - nes_debug(NES_DBG_CM, "Failed to get a Free pkt\n"); - return -1; - } - - /* send an MPA Request frame */ - form_cm_frame(skb, cm_node, NULL, 0, mpa_frame, mpa_frame_size, SET_ACK | SET_FIN); - ret = schedule_nes_timer(cm_node, skb, NES_TIMER_TYPE_SEND, 1, 0); + nes_debug(NES_DBG_CM, "%s cm_node=%p type=%d state=%d\n", + __func__, cm_node, cm_node->tcp_cntxt.client, cm_node->state); + if (cm_node->tcp_cntxt.client) + return ret; + cleanup_retrans_entry(cm_node); cm_node->state = NES_CM_STATE_CLOSED; ret = send_fin(cm_node, NULL); - if (ret < 0) { - printk(KERN_INFO PFX "failed to send MPA Reply (reject)\n"); - return ret; + if (cm_node->accept_pend) { + BUG_ON(!cm_node->listener); + atomic_dec(&cm_node->listener->pend_accepts_cnt); + BUG_ON(atomic_read(&cm_node->listener->pend_accepts_cnt) < 0); } + ret = send_reset(cm_node, NULL); return ret; } @@ -1783,35 +1980,39 @@ static int mini_cm_close(struct nes_cm_core *cm_core, struct nes_cm_node *cm_nod return -EINVAL; switch (cm_node->state) { - /* if passed in node is null, create a reference key node for node search */ - /* check if we found an owner node for this pkt */ - case NES_CM_STATE_SYN_RCVD: - case NES_CM_STATE_SYN_SENT: - case NES_CM_STATE_ONE_SIDE_ESTABLISHED: - case NES_CM_STATE_ESTABLISHED: - case NES_CM_STATE_ACCEPTING: - case NES_CM_STATE_MPAREQ_SENT: - cm_node->state = NES_CM_STATE_FIN_WAIT1; - send_fin(cm_node, NULL); - break; - case NES_CM_STATE_CLOSE_WAIT: - cm_node->state = NES_CM_STATE_LAST_ACK; - send_fin(cm_node, NULL); - break; - case NES_CM_STATE_FIN_WAIT1: - case NES_CM_STATE_FIN_WAIT2: - case NES_CM_STATE_LAST_ACK: - case NES_CM_STATE_TIME_WAIT: - case NES_CM_STATE_CLOSING: - ret = -1; - break; - case NES_CM_STATE_LISTENING: - case NES_CM_STATE_UNKNOWN: - case NES_CM_STATE_INITED: - case NES_CM_STATE_CLOSED: - case NES_CM_STATE_TSA: - ret = rem_ref_cm_node(cm_core, cm_node); - break; + case NES_CM_STATE_SYN_RCVD: + case NES_CM_STATE_SYN_SENT: + case NES_CM_STATE_ONE_SIDE_ESTABLISHED: + case NES_CM_STATE_ESTABLISHED: + case NES_CM_STATE_ACCEPTING: + case NES_CM_STATE_MPAREQ_SENT: + case NES_CM_STATE_MPAREQ_RCVD: + cleanup_retrans_entry(cm_node); + send_reset(cm_node, NULL); + break; + case NES_CM_STATE_CLOSE_WAIT: + cm_node->state = NES_CM_STATE_LAST_ACK; + send_fin(cm_node, NULL); + break; + case NES_CM_STATE_FIN_WAIT1: + case NES_CM_STATE_FIN_WAIT2: + case NES_CM_STATE_LAST_ACK: + case NES_CM_STATE_TIME_WAIT: + case NES_CM_STATE_CLOSING: + ret = -1; + break; + case NES_CM_STATE_LISTENING: + case NES_CM_STATE_UNKNOWN: + case NES_CM_STATE_INITED: + case NES_CM_STATE_CLOSED: + ret = rem_ref_cm_node(cm_core, cm_node); + break; + case NES_CM_STATE_TSA: + if (cm_node->send_entry) + printk(KERN_ERR "ERROR Close got called from STATE_TSA " + "send_entry=%p\n", cm_node->send_entry); + ret = rem_ref_cm_node(cm_core, cm_node); + break; } cm_node->cm_id = NULL; return ret; @@ -1822,25 +2023,30 @@ static int mini_cm_close(struct nes_cm_core *cm_core, struct nes_cm_node *cm_nod * recv_pkt - recv an ETHERNET packet, and process it through CM * node state machine */ -static int mini_cm_recv_pkt(struct nes_cm_core *cm_core, struct nes_vnic *nesvnic, - struct sk_buff *skb) +static void mini_cm_recv_pkt(struct nes_cm_core *cm_core, + struct nes_vnic *nesvnic, struct sk_buff *skb) { struct nes_cm_node *cm_node = NULL; struct nes_cm_listener *listener = NULL; struct iphdr *iph; struct tcphdr *tcph; struct nes_cm_info nfo; - int ret = 0; - if (!skb || skb->len < sizeof(struct iphdr) + sizeof(struct tcphdr)) { - ret = -EINVAL; - goto out; + if (!skb) + return; + if (skb->len < sizeof(struct iphdr) + sizeof(struct tcphdr)) { + dev_kfree_skb_any(skb); + return; } iph = (struct iphdr *)skb->data; tcph = (struct tcphdr *)(skb->data + sizeof(struct iphdr)); skb_reset_network_header(skb); skb_set_transport_header(skb, sizeof(*tcph)); + if (!tcph) { + dev_kfree_skb_any(skb); + return; + } skb->len = ntohs(iph->tot_len); nfo.loc_addr = ntohl(iph->daddr); @@ -1853,61 +2059,60 @@ static int mini_cm_recv_pkt(struct nes_cm_core *cm_core, struct nes_vnic *nesvni NIPQUAD(iph->daddr), tcph->dest, NIPQUAD(iph->saddr), tcph->source); - /* note: this call is going to increment cm_node ref count */ - cm_node = find_node(cm_core, + do { + cm_node = find_node(cm_core, nfo.rem_port, nfo.rem_addr, nfo.loc_port, nfo.loc_addr); - if (!cm_node) { - listener = find_listener(cm_core, nfo.loc_addr, nfo.loc_port, - NES_CM_LISTENER_ACTIVE_STATE); - if (listener) { - nfo.cm_id = listener->cm_id; - nfo.conn_type = listener->conn_type; - } else { - nfo.cm_id = NULL; - nfo.conn_type = 0; - } - - cm_node = make_cm_node(cm_core, nesvnic, &nfo, listener); if (!cm_node) { - nes_debug(NES_DBG_CM, "Unable to allocate node\n"); + /* Only type of packet accepted are for */ + /* the PASSIVE open (syn only) */ + if ((!tcph->syn) || (tcph->ack)) { + cm_packets_dropped++; + break; + } + listener = find_listener(cm_core, nfo.loc_addr, + nfo.loc_port, + NES_CM_LISTENER_ACTIVE_STATE); if (listener) { - nes_debug(NES_DBG_CM, "unable to allocate node and decrementing listener refcount\n"); + nfo.cm_id = listener->cm_id; + nfo.conn_type = listener->conn_type; + } else { + nes_debug(NES_DBG_CM, "Unable to find listener " + "for the pkt\n"); + cm_packets_dropped++; + dev_kfree_skb_any(skb); + break; + } + + cm_node = make_cm_node(cm_core, nesvnic, &nfo, + listener); + if (!cm_node) { + nes_debug(NES_DBG_CM, "Unable to allocate " + "node\n"); + cm_packets_dropped++; atomic_dec(&listener->ref_count); + dev_kfree_skb_any(skb); + break; } - ret = -1; - goto out; - } - if (!listener) { - nes_debug(NES_DBG_CM, "Packet found for unknown port %x refcnt=%d\n", - nfo.loc_port, atomic_read(&cm_node->ref_count)); - if (!tcph->rst) { - nes_debug(NES_DBG_CM, "Packet found for unknown port=%d" - " rem_port=%d refcnt=%d\n", - nfo.loc_port, nfo.rem_port, atomic_read(&cm_node->ref_count)); - - cm_node->tcp_cntxt.rcv_nxt = ntohl(tcph->seq); - cm_node->tcp_cntxt.loc_seq_num = ntohl(tcph->ack_seq); - send_reset(cm_node); + if (!tcph->rst && !tcph->fin) { + cm_node->state = NES_CM_STATE_LISTENING; + } else { + cm_packets_dropped++; + rem_ref_cm_node(cm_core, cm_node); + dev_kfree_skb_any(skb); + break; } + add_ref_cm_node(cm_node); + } else if (cm_node->state == NES_CM_STATE_TSA) { rem_ref_cm_node(cm_core, cm_node); - ret = -1; - goto out; + atomic_inc(&cm_accel_dropped_pkts); + dev_kfree_skb_any(skb); + break; } - add_ref_cm_node(cm_node); - cm_node->state = NES_CM_STATE_LISTENING; - } - - nes_debug(NES_DBG_CM, "Processing Packet for node %p, data = (%p):\n", - cm_node, skb->data); - process_packet(cm_node, skb, cm_core); - - rem_ref_cm_node(cm_core, cm_node); - out: - if (skb) - dev_kfree_skb_any(skb); - return ret; + process_packet(cm_node, skb, cm_core); + rem_ref_cm_node(cm_core, cm_node); + } while (0); } @@ -2107,15 +2312,12 @@ int nes_cm_disconn(struct nes_qp *nesqp) if (nesqp->disconn_pending == 0) { nesqp->disconn_pending++; spin_unlock_irqrestore(&nesqp->lock, flags); - /* nes_add_ref(&nesqp->ibqp); */ /* init our disconnect work element, to */ INIT_WORK(&nesqp->disconn_work, nes_disconnect_worker); queue_work(g_cm_core->disconn_wq, &nesqp->disconn_work); - } else { + } else spin_unlock_irqrestore(&nesqp->lock, flags); - nes_rem_ref(&nesqp->ibqp); - } return 0; } @@ -2161,7 +2363,6 @@ static int nes_cm_disconn_true(struct nes_qp *nesqp) nes_debug(NES_DBG_CM, "QP%u disconnect_worker cmid is NULL\n", nesqp->hwqp.qp_id); spin_unlock_irqrestore(&nesqp->lock, flags); - nes_rem_ref(&nesqp->ibqp); return -1; } @@ -2182,30 +2383,31 @@ static int nes_cm_disconn_true(struct nes_qp *nesqp) atomic_inc(&cm_disconnects); cm_event.event = IW_CM_EVENT_DISCONNECT; if (last_ae == NES_AEQE_AEID_LLP_CONNECTION_RESET) { - issued_disconnect_reset = 1; cm_event.status = IW_CM_EVENT_STATUS_RESET; - nes_debug(NES_DBG_CM, "Generating a CM Disconnect Event (status reset) for " - " QP%u, cm_id = %p. \n", - nesqp->hwqp.qp_id, cm_id); - } else { + nes_debug(NES_DBG_CM, "Generating a CM " + "Disconnect Event (status reset) for " + "QP%u, cm_id = %p. \n", + nesqp->hwqp.qp_id, cm_id); + } else cm_event.status = IW_CM_EVENT_STATUS_OK; - } cm_event.local_addr = cm_id->local_addr; cm_event.remote_addr = cm_id->remote_addr; cm_event.private_data = NULL; cm_event.private_data_len = 0; - nes_debug(NES_DBG_CM, "Generating a CM Disconnect Event for " - " QP%u, SQ Head = %u, SQ Tail = %u. cm_id = %p, refcount = %u.\n", - nesqp->hwqp.qp_id, - nesqp->hwqp.sq_head, nesqp->hwqp.sq_tail, cm_id, - atomic_read(&nesqp->refcount)); + nes_debug(NES_DBG_CM, "Generating a CM Disconnect Event" + " for QP%u, SQ Head = %u, SQ Tail = %u. " + "cm_id = %p, refcount = %u.\n", + nesqp->hwqp.qp_id, nesqp->hwqp.sq_head, + nesqp->hwqp.sq_tail, cm_id, + atomic_read(&nesqp->refcount)); spin_unlock_irqrestore(&nesqp->lock, flags); ret = cm_id->event_handler(cm_id, &cm_event); if (ret) - nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret); + nes_debug(NES_DBG_CM, "OFA CM event_handler " + "returned, ret=%d\n", ret); spin_lock_irqsave(&nesqp->lock, flags); } @@ -2247,31 +2449,24 @@ static int nes_cm_disconn_true(struct nes_qp *nesqp) if (nesqp->flush_issued == 0) { nesqp->flush_issued = 1; spin_unlock_irqrestore(&nesqp->lock, flags); - flush_wqes(nesvnic->nesdev, nesqp, NES_CQP_FLUSH_RQ, 1); - } else { + flush_wqes(nesvnic->nesdev, nesqp, + NES_CQP_FLUSH_RQ, 1); + } else spin_unlock_irqrestore(&nesqp->lock, flags); - } - - /* This reference is from either ModifyQP or the AE processing, - there is still a race here with modifyqp */ - nes_rem_ref(&nesqp->ibqp); - } else { cm_id = nesqp->cm_id; spin_unlock_irqrestore(&nesqp->lock, flags); /* check to see if the inbound reset beat the outbound reset */ if ((!cm_id) && (last_ae==NES_AEQE_AEID_RESET_SENT)) { - nes_debug(NES_DBG_CM, "QP%u: Decing refcount due to inbound reset" - " beating the outbound reset.\n", - nesqp->hwqp.qp_id); - nes_rem_ref(&nesqp->ibqp); + nes_debug(NES_DBG_CM, "QP%u: Decing refcount " + "due to inbound reset beating the " + "outbound reset.\n", nesqp->hwqp.qp_id); } } } else { nesqp->disconn_pending = 0; spin_unlock_irqrestore(&nesqp->lock, flags); } - nes_rem_ref(&nesqp->ibqp); return 0; } @@ -2349,71 +2544,82 @@ int nes_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) nesdev = nesvnic->nesdev; adapter = nesdev->nesadapter; - nes_debug(NES_DBG_CM, "nesvnic=%p, netdev=%p, %s\n", - nesvnic, nesvnic->netdev, nesvnic->netdev->name); - - /* since this is from a listen, we were able to put node handle into cm_id */ cm_node = (struct nes_cm_node *)cm_id->provider_data; + nes_debug(NES_DBG_CM, "nes_accept: cm_node= %p nesvnic=%p, netdev=%p," + "%s\n", cm_node, nesvnic, nesvnic->netdev, + nesvnic->netdev->name); /* associate the node with the QP */ nesqp->cm_node = (void *)cm_node; + cm_node->nesqp = nesqp; + nes_add_ref(&nesqp->ibqp); - nes_debug(NES_DBG_CM, "QP%u, cm_node=%p, jiffies = %lu\n", - nesqp->hwqp.qp_id, cm_node, jiffies); + nes_debug(NES_DBG_CM, "QP%u, cm_node=%p, jiffies = %lu listener = %p\n", + nesqp->hwqp.qp_id, cm_node, jiffies, cm_node->listener); atomic_inc(&cm_accepts); nes_debug(NES_DBG_CM, "netdev refcnt = %u.\n", atomic_read(&nesvnic->netdev->refcnt)); - /* allocate the ietf frame and space for private data */ - nesqp->ietf_frame = pci_alloc_consistent(nesdev->pcidev, - sizeof(struct ietf_mpa_frame) + conn_param->private_data_len, - &nesqp->ietf_frame_pbase); - - if (!nesqp->ietf_frame) { - nes_debug(NES_DBG_CM, "Unable to allocate memory for private data\n"); - return -ENOMEM; - } + /* allocate the ietf frame and space for private data */ + nesqp->ietf_frame = pci_alloc_consistent(nesdev->pcidev, + sizeof(struct ietf_mpa_frame) + conn_param->private_data_len, + &nesqp->ietf_frame_pbase); + if (!nesqp->ietf_frame) { + nes_debug(NES_DBG_CM, "Unable to allocate memory for private " + "data\n"); + return -ENOMEM; + } - /* setup the MPA frame */ - nesqp->private_data_len = conn_param->private_data_len; - memcpy(nesqp->ietf_frame->key, IEFT_MPA_KEY_REP, IETF_MPA_KEY_SIZE); - memcpy(nesqp->ietf_frame->priv_data, conn_param->private_data, - conn_param->private_data_len); + /* setup the MPA frame */ + nesqp->private_data_len = conn_param->private_data_len; + memcpy(nesqp->ietf_frame->key, IEFT_MPA_KEY_REP, IETF_MPA_KEY_SIZE); - nesqp->ietf_frame->priv_data_len = cpu_to_be16(conn_param->private_data_len); - nesqp->ietf_frame->rev = mpa_version; - nesqp->ietf_frame->flags = IETF_MPA_FLAGS_CRC; + memcpy(nesqp->ietf_frame->priv_data, conn_param->private_data, + conn_param->private_data_len); - /* setup our first outgoing iWarp send WQE (the IETF frame response) */ - wqe = &nesqp->hwqp.sq_vbase[0]; + nesqp->ietf_frame->priv_data_len = + cpu_to_be16(conn_param->private_data_len); + nesqp->ietf_frame->rev = mpa_version; + nesqp->ietf_frame->flags = IETF_MPA_FLAGS_CRC; - if (cm_id->remote_addr.sin_addr.s_addr != cm_id->local_addr.sin_addr.s_addr) { - u64temp = (unsigned long)nesqp; - u64temp |= NES_SW_CONTEXT_ALIGN>>1; - set_wqe_64bit_value(wqe->wqe_words, NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, - u64temp); - wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = - cpu_to_le32(NES_IWARP_SQ_WQE_STREAMING | NES_IWARP_SQ_WQE_WRPDU); - wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = - cpu_to_le32(conn_param->private_data_len + sizeof(struct ietf_mpa_frame)); - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = - cpu_to_le32((u32)nesqp->ietf_frame_pbase); - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = - cpu_to_le32((u32)((u64)nesqp->ietf_frame_pbase >> 32)); - wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = - cpu_to_le32(conn_param->private_data_len + sizeof(struct ietf_mpa_frame)); - wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; - - nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32( - NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | NES_QPCONTEXT_ORDIRD_WRPDU); - } else { - nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32((NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | - NES_QPCONTEXT_ORDIRD_WRPDU | NES_QPCONTEXT_ORDIRD_ALSMM)); - } - nesqp->skip_lsmm = 1; + /* setup our first outgoing iWarp send WQE (the IETF frame response) */ + wqe = &nesqp->hwqp.sq_vbase[0]; + + if (cm_id->remote_addr.sin_addr.s_addr != + cm_id->local_addr.sin_addr.s_addr) { + u64temp = (unsigned long)nesqp; + u64temp |= NES_SW_CONTEXT_ALIGN>>1; + set_wqe_64bit_value(wqe->wqe_words, + NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, + u64temp); + wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = + cpu_to_le32(NES_IWARP_SQ_WQE_STREAMING | + NES_IWARP_SQ_WQE_WRPDU); + wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = + cpu_to_le32(conn_param->private_data_len + + sizeof(struct ietf_mpa_frame)); + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = + cpu_to_le32((u32)nesqp->ietf_frame_pbase); + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = + cpu_to_le32((u32)((u64)nesqp->ietf_frame_pbase >> 32)); + wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = + cpu_to_le32(conn_param->private_data_len + + sizeof(struct ietf_mpa_frame)); + wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; + + nesqp->nesqp_context->ird_ord_sizes |= + cpu_to_le32(NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | + NES_QPCONTEXT_ORDIRD_WRPDU); + } else { + nesqp->nesqp_context->ird_ord_sizes |= + cpu_to_le32((NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | + NES_QPCONTEXT_ORDIRD_WRPDU | + NES_QPCONTEXT_ORDIRD_ALSMM)); + } + nesqp->skip_lsmm = 1; /* Cache the cm_id in the qp */ @@ -2424,55 +2630,75 @@ int nes_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) cm_id->provider_data = nesqp; nesqp->active_conn = 0; + if (cm_node->state == NES_CM_STATE_TSA) + nes_debug(NES_DBG_CM, "Already state = TSA for cm_node=%p\n", + cm_node); + nes_cm_init_tsa_conn(nesqp, cm_node); - nesqp->nesqp_context->tcpPorts[0] = cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); - nesqp->nesqp_context->tcpPorts[1] = cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); - nesqp->nesqp_context->ip0 = cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); + nesqp->nesqp_context->tcpPorts[0] = + cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); + nesqp->nesqp_context->tcpPorts[1] = + cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); + + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(nesvnic->local_ipaddr)); + else + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); nesqp->nesqp_context->misc2 |= cpu_to_le32( - (u32)PCI_FUNC(nesdev->pcidev->devfn) << NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); + (u32)PCI_FUNC(nesdev->pcidev->devfn) << + NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); - nesqp->nesqp_context->arp_index_vlan |= cpu_to_le32( - nes_arp_table(nesdev, le32_to_cpu(nesqp->nesqp_context->ip0), NULL, + nesqp->nesqp_context->arp_index_vlan |= + cpu_to_le32(nes_arp_table(nesdev, + le32_to_cpu(nesqp->nesqp_context->ip0), NULL, NES_ARP_RESOLVE) << 16); nesqp->nesqp_context->ts_val_delta = cpu_to_le32( - jiffies - nes_read_indexed(nesdev, NES_IDX_TCP_NOW)); + jiffies - nes_read_indexed(nesdev, NES_IDX_TCP_NOW)); nesqp->nesqp_context->ird_index = cpu_to_le32(nesqp->hwqp.qp_id); nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32( - ((u32)1 << NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT)); - nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32((u32)conn_param->ord); + ((u32)1 << NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT)); + nesqp->nesqp_context->ird_ord_sizes |= + cpu_to_le32((u32)conn_param->ord); memset(&nes_quad, 0, sizeof(nes_quad)); - nes_quad.DstIpAdrIndex = cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); - nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; - nes_quad.TcpPorts[0] = cm_id->remote_addr.sin_port; - nes_quad.TcpPorts[1] = cm_id->local_addr.sin_port; + nes_quad.DstIpAdrIndex = + cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nes_quad.SrcIpadr = nesvnic->local_ipaddr; + else + nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; + nes_quad.TcpPorts[0] = cm_id->remote_addr.sin_port; + nes_quad.TcpPorts[1] = cm_id->local_addr.sin_port; /* Produce hash key */ crc_value = get_crc_value(&nes_quad); nesqp->hte_index = cpu_to_be32(crc_value ^ 0xffffffff); nes_debug(NES_DBG_CM, "HTE Index = 0x%08X, CRC = 0x%08X\n", - nesqp->hte_index, nesqp->hte_index & adapter->hte_index_mask); + nesqp->hte_index, nesqp->hte_index & adapter->hte_index_mask); nesqp->hte_index &= adapter->hte_index_mask; nesqp->nesqp_context->hte_index = cpu_to_le32(nesqp->hte_index); cm_node->cm_core->api->accelerated(cm_node->cm_core, cm_node); - nes_debug(NES_DBG_CM, "QP%u, Destination IP = 0x%08X:0x%04X, local = 0x%08X:0x%04X," - " rcv_nxt=0x%08X, snd_nxt=0x%08X, mpa + private data length=%zu.\n", - nesqp->hwqp.qp_id, + nes_debug(NES_DBG_CM, "QP%u, Destination IP = 0x%08X:0x%04X, local = " + "0x%08X:0x%04X, rcv_nxt=0x%08X, snd_nxt=0x%08X, mpa + " + "private data length=%zu.\n", nesqp->hwqp.qp_id, ntohl(cm_id->remote_addr.sin_addr.s_addr), ntohs(cm_id->remote_addr.sin_port), ntohl(cm_id->local_addr.sin_addr.s_addr), ntohs(cm_id->local_addr.sin_port), le32_to_cpu(nesqp->nesqp_context->rcv_nxt), le32_to_cpu(nesqp->nesqp_context->snd_nxt), - conn_param->private_data_len+sizeof(struct ietf_mpa_frame)); + conn_param->private_data_len + + sizeof(struct ietf_mpa_frame)); attr.qp_state = IB_QPS_RTS; nes_modify_qp(&nesqp->ibqp, &attr, IB_QP_STATE, NULL); @@ -2489,15 +2715,16 @@ int nes_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) cm_event.private_data_len = 0; ret = cm_id->event_handler(cm_id, &cm_event); if (cm_node->loopbackpartner) { - cm_node->loopbackpartner->mpa_frame_size = nesqp->private_data_len; + cm_node->loopbackpartner->mpa_frame_size = + nesqp->private_data_len; /* copy entire MPA frame to our cm_node's frame */ - memcpy(cm_node->loopbackpartner->mpa_frame_buf, nesqp->ietf_frame->priv_data, - nesqp->private_data_len); + memcpy(cm_node->loopbackpartner->mpa_frame_buf, + nesqp->ietf_frame->priv_data, nesqp->private_data_len); create_event(cm_node->loopbackpartner, NES_CM_EVENT_CONNECTED); } if (ret) - printk("%s[%u] OFA CM event_handler returned, ret=%d\n", - __func__, __LINE__, ret); + printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " + "ret=%d\n", __func__, __LINE__, ret); return 0; } @@ -2555,74 +2782,61 @@ int nes_connect(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) if (!nesdev) return -EINVAL; - atomic_inc(&cm_connects); - - nesqp->ietf_frame = kzalloc(sizeof(struct ietf_mpa_frame) + - conn_param->private_data_len, GFP_KERNEL); - if (!nesqp->ietf_frame) - return -ENOMEM; + nes_debug(NES_DBG_CM, "QP%u, current IP = 0x%08X, Destination IP = " + "0x%08X:0x%04X, local = 0x%08X:0x%04X.\n", nesqp->hwqp.qp_id, + ntohl(nesvnic->local_ipaddr), + ntohl(cm_id->remote_addr.sin_addr.s_addr), + ntohs(cm_id->remote_addr.sin_port), + ntohl(cm_id->local_addr.sin_addr.s_addr), + ntohs(cm_id->local_addr.sin_port)); - /* set qp as having an active connection */ + atomic_inc(&cm_connects); nesqp->active_conn = 1; - nes_debug(NES_DBG_CM, "QP%u, Destination IP = 0x%08X:0x%04X, local = 0x%08X:0x%04X.\n", - nesqp->hwqp.qp_id, - ntohl(cm_id->remote_addr.sin_addr.s_addr), - ntohs(cm_id->remote_addr.sin_port), - ntohl(cm_id->local_addr.sin_addr.s_addr), - ntohs(cm_id->local_addr.sin_port)); - /* cache the cm_id in the qp */ nesqp->cm_id = cm_id; cm_id->provider_data = nesqp; - /* copy the private data */ - if (conn_param->private_data_len) { - memcpy(nesqp->ietf_frame->priv_data, conn_param->private_data, - conn_param->private_data_len); - } - nesqp->private_data_len = conn_param->private_data_len; nesqp->nesqp_context->ird_ord_sizes |= cpu_to_le32((u32)conn_param->ord); nes_debug(NES_DBG_CM, "requested ord = 0x%08X.\n", (u32)conn_param->ord); - nes_debug(NES_DBG_CM, "mpa private data len =%u\n", conn_param->private_data_len); - - strcpy(&nesqp->ietf_frame->key[0], IEFT_MPA_KEY_REQ); - nesqp->ietf_frame->flags = IETF_MPA_FLAGS_CRC; - nesqp->ietf_frame->rev = IETF_MPA_VERSION; - nesqp->ietf_frame->priv_data_len = htons(conn_param->private_data_len); + nes_debug(NES_DBG_CM, "mpa private data len =%u\n", + conn_param->private_data_len); - if (cm_id->local_addr.sin_addr.s_addr != cm_id->remote_addr.sin_addr.s_addr) + if (cm_id->local_addr.sin_addr.s_addr != + cm_id->remote_addr.sin_addr.s_addr) nes_manage_apbvt(nesvnic, ntohs(cm_id->local_addr.sin_port), - PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD); + PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD); /* set up the connection params for the node */ - cm_info.loc_addr = (cm_id->local_addr.sin_addr.s_addr); - cm_info.loc_port = (cm_id->local_addr.sin_port); - cm_info.rem_addr = (cm_id->remote_addr.sin_addr.s_addr); - cm_info.rem_port = (cm_id->remote_addr.sin_port); + cm_info.loc_addr = htonl(cm_id->local_addr.sin_addr.s_addr); + cm_info.loc_port = htons(cm_id->local_addr.sin_port); + cm_info.rem_addr = htonl(cm_id->remote_addr.sin_addr.s_addr); + cm_info.rem_port = htons(cm_id->remote_addr.sin_port); cm_info.cm_id = cm_id; cm_info.conn_type = NES_CM_IWARP_CONN_TYPE; cm_id->add_ref(cm_id); - nes_add_ref(&nesqp->ibqp); /* create a connect CM node connection */ - cm_node = g_cm_core->api->connect(g_cm_core, nesvnic, nesqp->ietf_frame, &cm_info); + cm_node = g_cm_core->api->connect(g_cm_core, nesvnic, + conn_param->private_data_len, (void *)conn_param->private_data, + &cm_info); if (!cm_node) { - if (cm_id->local_addr.sin_addr.s_addr != cm_id->remote_addr.sin_addr.s_addr) + if (cm_id->local_addr.sin_addr.s_addr != + cm_id->remote_addr.sin_addr.s_addr) nes_manage_apbvt(nesvnic, ntohs(cm_id->local_addr.sin_port), - PCI_FUNC(nesdev->pcidev->devfn), NES_MANAGE_APBVT_DEL); - nes_rem_ref(&nesqp->ibqp); - kfree(nesqp->ietf_frame); - nesqp->ietf_frame = NULL; + PCI_FUNC(nesdev->pcidev->devfn), + NES_MANAGE_APBVT_DEL); + cm_id->rem_ref(cm_id); return -ENOMEM; } cm_node->apbvt_set = 1; nesqp->cm_node = cm_node; + cm_node->nesqp = nesqp; return 0; } @@ -2664,7 +2878,7 @@ int nes_create_listen(struct iw_cm_id *cm_id, int backlog) cm_node = g_cm_core->api->listen(g_cm_core, nesvnic, &cm_info); if (!cm_node) { - printk("%s[%u] Error returned from listen API call\n", + printk(KERN_ERR "%s[%u] Error returned from listen API call\n", __func__, __LINE__); return -ENOMEM; } @@ -2672,10 +2886,13 @@ int nes_create_listen(struct iw_cm_id *cm_id, int backlog) cm_id->provider_data = cm_node; if (!cm_node->reused_node) { - err = nes_manage_apbvt(nesvnic, ntohs(cm_id->local_addr.sin_port), - PCI_FUNC(nesvnic->nesdev->pcidev->devfn), NES_MANAGE_APBVT_ADD); + err = nes_manage_apbvt(nesvnic, + ntohs(cm_id->local_addr.sin_port), + PCI_FUNC(nesvnic->nesdev->pcidev->devfn), + NES_MANAGE_APBVT_ADD); if (err) { - printk("nes_manage_apbvt call returned %d.\n", err); + printk(KERN_ERR "nes_manage_apbvt call returned %d.\n", + err); g_cm_core->api->stop_listener(g_cm_core, (void *)cm_node); return err; } @@ -2795,53 +3012,70 @@ static void cm_event_connected(struct nes_cm_event *event) nes_cm_init_tsa_conn(nesqp, cm_node); /* set the QP tsa context */ - nesqp->nesqp_context->tcpPorts[0] = cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); - nesqp->nesqp_context->tcpPorts[1] = cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); - nesqp->nesqp_context->ip0 = cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); + nesqp->nesqp_context->tcpPorts[0] = + cpu_to_le16(ntohs(cm_id->local_addr.sin_port)); + nesqp->nesqp_context->tcpPorts[1] = + cpu_to_le16(ntohs(cm_id->remote_addr.sin_port)); + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(nesvnic->local_ipaddr)); + else + nesqp->nesqp_context->ip0 = + cpu_to_le32(ntohl(cm_id->remote_addr.sin_addr.s_addr)); nesqp->nesqp_context->misc2 |= cpu_to_le32( - (u32)PCI_FUNC(nesdev->pcidev->devfn) << NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); + (u32)PCI_FUNC(nesdev->pcidev->devfn) << + NES_QPCONTEXT_MISC2_SRC_IP_SHIFT); nesqp->nesqp_context->arp_index_vlan |= cpu_to_le32( - nes_arp_table(nesdev, le32_to_cpu(nesqp->nesqp_context->ip0), + nes_arp_table(nesdev, + le32_to_cpu(nesqp->nesqp_context->ip0), NULL, NES_ARP_RESOLVE) << 16); nesqp->nesqp_context->ts_val_delta = cpu_to_le32( jiffies - nes_read_indexed(nesdev, NES_IDX_TCP_NOW)); nesqp->nesqp_context->ird_index = cpu_to_le32(nesqp->hwqp.qp_id); nesqp->nesqp_context->ird_ord_sizes |= - cpu_to_le32((u32)1 << NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT); + cpu_to_le32((u32)1 << + NES_QPCONTEXT_ORDIRD_IWARP_MODE_SHIFT); /* Adjust tail for not having a LSMM */ nesqp->hwqp.sq_tail = 1; #if defined(NES_SEND_FIRST_WRITE) - if (cm_node->send_write0) { - nes_debug(NES_DBG_CM, "Sending first write.\n"); - wqe = &nesqp->hwqp.sq_vbase[0]; - u64temp = (unsigned long)nesqp; - u64temp |= NES_SW_CONTEXT_ALIGN>>1; - set_wqe_64bit_value(wqe->wqe_words, NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, - u64temp); - wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = cpu_to_le32(NES_IWARP_SQ_OP_RDMAW); - wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = 0; - wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; - - /* use the reserved spot on the WQ for the extra first WQE */ - nesqp->nesqp_context->ird_ord_sizes &= cpu_to_le32(~(NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | - NES_QPCONTEXT_ORDIRD_WRPDU | NES_QPCONTEXT_ORDIRD_ALSMM)); - nesqp->skip_lsmm = 1; - nesqp->hwqp.sq_tail = 0; - nes_write32(nesdev->regs + NES_WQE_ALLOC, - (1 << 24) | 0x00800000 | nesqp->hwqp.qp_id); - } + if (cm_node->send_write0) { + nes_debug(NES_DBG_CM, "Sending first write.\n"); + wqe = &nesqp->hwqp.sq_vbase[0]; + u64temp = (unsigned long)nesqp; + u64temp |= NES_SW_CONTEXT_ALIGN>>1; + set_wqe_64bit_value(wqe->wqe_words, + NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX, u64temp); + wqe->wqe_words[NES_IWARP_SQ_WQE_MISC_IDX] = + cpu_to_le32(NES_IWARP_SQ_OP_RDMAW); + wqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_LOW_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_FRAG0_HIGH_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_LENGTH0_IDX] = 0; + wqe->wqe_words[NES_IWARP_SQ_WQE_STAG0_IDX] = 0; + + /* use the reserved spot on the WQ for the extra first WQE */ + nesqp->nesqp_context->ird_ord_sizes &= + cpu_to_le32(~(NES_QPCONTEXT_ORDIRD_LSMM_PRESENT | + NES_QPCONTEXT_ORDIRD_WRPDU | + NES_QPCONTEXT_ORDIRD_ALSMM)); + nesqp->skip_lsmm = 1; + nesqp->hwqp.sq_tail = 0; + nes_write32(nesdev->regs + NES_WQE_ALLOC, + (1 << 24) | 0x00800000 | nesqp->hwqp.qp_id); + } #endif memset(&nes_quad, 0, sizeof(nes_quad)); - nes_quad.DstIpAdrIndex = cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); - nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; + nes_quad.DstIpAdrIndex = + cpu_to_le32((u32)PCI_FUNC(nesdev->pcidev->devfn) << 24); + if (ipv4_is_loopback(cm_id->remote_addr.sin_addr.s_addr)) + nes_quad.SrcIpadr = nesvnic->local_ipaddr; + else + nes_quad.SrcIpadr = cm_id->remote_addr.sin_addr.s_addr; nes_quad.TcpPorts[0] = cm_id->remote_addr.sin_port; nes_quad.TcpPorts[1] = cm_id->local_addr.sin_port; @@ -2858,10 +3092,6 @@ static void cm_event_connected(struct nes_cm_event *event) nesqp->private_data_len = (u8) cm_node->mpa_frame_size; cm_node->cm_core->api->accelerated(cm_node->cm_core, cm_node); - /* modify QP state to rts */ - attr.qp_state = IB_QPS_RTS; - nes_modify_qp(&nesqp->ibqp, &attr, IB_QP_STATE, NULL); - /* notify OF layer we successfully created the requested connection */ cm_event.event = IW_CM_EVENT_CONNECT_REPLY; cm_event.status = IW_CM_EVENT_STATUS_ACCEPTED; @@ -2870,20 +3100,21 @@ static void cm_event_connected(struct nes_cm_event *event) cm_event.local_addr.sin_port = cm_id->local_addr.sin_port; cm_event.remote_addr = cm_id->remote_addr; - cm_event.private_data = (void *)event->cm_node->mpa_frame_buf; - cm_event.private_data_len = (u8) event->cm_node->mpa_frame_size; + cm_event.private_data = (void *)event->cm_node->mpa_frame_buf; + cm_event.private_data_len = (u8) event->cm_node->mpa_frame_size; cm_event.local_addr.sin_addr.s_addr = event->cm_info.rem_addr; ret = cm_id->event_handler(cm_id, &cm_event); nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret); if (ret) - printk("%s[%u] OFA CM event_handler returned, ret=%d\n", - __func__, __LINE__, ret); - nes_debug(NES_DBG_CM, "Exiting connect thread for QP%u. jiffies = %lu\n", - nesqp->hwqp.qp_id, jiffies ); + printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " + "ret=%d\n", __func__, __LINE__, ret); + attr.qp_state = IB_QPS_RTS; + nes_modify_qp(&nesqp->ibqp, &attr, IB_QP_STATE, NULL); - nes_rem_ref(&nesqp->ibqp); + nes_debug(NES_DBG_CM, "Exiting connect thread for QP%u. jiffies = " + "%lu\n", nesqp->hwqp.qp_id, jiffies); return; } @@ -2927,17 +3158,19 @@ static void cm_event_connect_error(struct nes_cm_event *event) cm_event.private_data = NULL; cm_event.private_data_len = 0; - nes_debug(NES_DBG_CM, "call CM_EVENT REJECTED, local_addr=%08x, remove_addr=%08x\n", - cm_event.local_addr.sin_addr.s_addr, cm_event.remote_addr.sin_addr.s_addr); + nes_debug(NES_DBG_CM, "call CM_EVENT REJECTED, local_addr=%08x, " + "remove_addr=%08x\n", cm_event.local_addr.sin_addr.s_addr, + cm_event.remote_addr.sin_addr.s_addr); ret = cm_id->event_handler(cm_id, &cm_event); nes_debug(NES_DBG_CM, "OFA CM event_handler returned, ret=%d\n", ret); if (ret) - printk("%s[%u] OFA CM event_handler returned, ret=%d\n", - __func__, __LINE__, ret); + printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " + "ret=%d\n", __func__, __LINE__, ret); nes_rem_ref(&nesqp->ibqp); - cm_id->rem_ref(cm_id); + cm_id->rem_ref(cm_id); + rem_ref_cm_node(event->cm_node->cm_core, event->cm_node); return; } @@ -3040,7 +3273,8 @@ static int nes_cm_post_event(struct nes_cm_event *event) add_ref_cm_node(event->cm_node); event->cm_info.cm_id->add_ref(event->cm_info.cm_id); INIT_WORK(&event->event_work, nes_cm_event_handler); - nes_debug(NES_DBG_CM, "queue_work, event=%p\n", event); + nes_debug(NES_DBG_CM, "cm_node=%p queue_work, event=%p\n", + event->cm_node, event); queue_work(event->cm_node->cm_core->event_wq, &event->event_work); @@ -3056,46 +3290,48 @@ static int nes_cm_post_event(struct nes_cm_event *event) */ static void nes_cm_event_handler(struct work_struct *work) { - struct nes_cm_event *event = container_of(work, struct nes_cm_event, event_work); + struct nes_cm_event *event = container_of(work, struct nes_cm_event, + event_work); struct nes_cm_core *cm_core; - if ((!event) || (!event->cm_node) || (!event->cm_node->cm_core)) { + if ((!event) || (!event->cm_node) || (!event->cm_node->cm_core)) return; - } + cm_core = event->cm_node->cm_core; nes_debug(NES_DBG_CM, "event=%p, event->type=%u, events posted=%u\n", - event, event->type, atomic_read(&cm_core->events_posted)); + event, event->type, atomic_read(&cm_core->events_posted)); switch (event->type) { - case NES_CM_EVENT_MPA_REQ: - cm_event_mpa_req(event); - nes_debug(NES_DBG_CM, "CM Event: MPA REQUEST\n"); - break; - case NES_CM_EVENT_RESET: - nes_debug(NES_DBG_CM, "CM Event: RESET\n"); - cm_event_reset(event); - break; - case NES_CM_EVENT_CONNECTED: - if ((!event->cm_node->cm_id) || - (event->cm_node->state != NES_CM_STATE_TSA)) { - break; - } - cm_event_connected(event); - nes_debug(NES_DBG_CM, "CM Event: CONNECTED\n"); + case NES_CM_EVENT_MPA_REQ: + cm_event_mpa_req(event); + nes_debug(NES_DBG_CM, "cm_node=%p CM Event: MPA REQUEST\n", + event->cm_node); + break; + case NES_CM_EVENT_RESET: + nes_debug(NES_DBG_CM, "cm_node = %p CM Event: RESET\n", + event->cm_node); + cm_event_reset(event); + break; + case NES_CM_EVENT_CONNECTED: + if ((!event->cm_node->cm_id) || + (event->cm_node->state != NES_CM_STATE_TSA)) break; - case NES_CM_EVENT_ABORTED: - if ((!event->cm_node->cm_id) || (event->cm_node->state == NES_CM_STATE_TSA)) { - break; - } - cm_event_connect_error(event); - nes_debug(NES_DBG_CM, "CM Event: ABORTED\n"); - break; - case NES_CM_EVENT_DROPPED_PKT: - nes_debug(NES_DBG_CM, "CM Event: DROPPED PKT\n"); - break; - default: - nes_debug(NES_DBG_CM, "CM Event: UNKNOWN EVENT TYPE\n"); + cm_event_connected(event); + nes_debug(NES_DBG_CM, "CM Event: CONNECTED\n"); + break; + case NES_CM_EVENT_ABORTED: + if ((!event->cm_node->cm_id) || + (event->cm_node->state == NES_CM_STATE_TSA)) break; + cm_event_connect_error(event); + nes_debug(NES_DBG_CM, "CM Event: ABORTED\n"); + break; + case NES_CM_EVENT_DROPPED_PKT: + nes_debug(NES_DBG_CM, "CM Event: DROPPED PKT\n"); + break; + default: + nes_debug(NES_DBG_CM, "CM Event: UNKNOWN EVENT TYPE\n"); + break; } atomic_dec(&cm_core->events_posted); diff --git a/drivers/infiniband/hw/nes/nes_cm.h b/drivers/infiniband/hw/nes/nes_cm.h index 7717cb2..367b3d2 100644 --- a/drivers/infiniband/hw/nes/nes_cm.h +++ b/drivers/infiniband/hw/nes/nes_cm.h @@ -83,6 +83,8 @@ enum nes_timer_type { #define SET_FIN 4 #define SET_RST 8 +#define TCP_OPTIONS_PADDING 3 + struct option_base { u8 optionnum; u8 length; @@ -177,6 +179,7 @@ enum nes_cm_node_state { NES_CM_STATE_ESTABLISHED, NES_CM_STATE_ACCEPTING, NES_CM_STATE_MPAREQ_SENT, + NES_CM_STATE_MPAREQ_RCVD, NES_CM_STATE_TSA, NES_CM_STATE_FIN_WAIT1, NES_CM_STATE_FIN_WAIT2, @@ -187,6 +190,16 @@ enum nes_cm_node_state { NES_CM_STATE_CLOSED }; +enum nes_tcpip_pkt_type { + NES_PKT_TYPE_UNKNOWN, + NES_PKT_TYPE_SYN, + NES_PKT_TYPE_SYNACK, + NES_PKT_TYPE_ACK, + NES_PKT_TYPE_FIN, + NES_PKT_TYPE_RST +}; + + /* type of nes connection */ enum nes_cm_conn_type { NES_CM_IWARP_CONN_TYPE, @@ -257,7 +270,9 @@ struct nes_cm_node { struct net_device *netdev; struct nes_cm_node *loopbackpartner; - struct list_head retrans_list; + + struct nes_timer_entry *send_entry; + spinlock_t retrans_list_lock; struct list_head recv_list; spinlock_t recv_list_lock; @@ -276,6 +291,8 @@ struct nes_cm_node { struct nes_vnic *nesvnic; int apbvt_set; int accept_pend; + int freed; + struct nes_qp *nesqp; }; /* structure for client or CM to fill when making CM api calls. */ @@ -366,14 +383,14 @@ struct nes_cm_ops { struct nes_cm_info *); int (*stop_listener)(struct nes_cm_core *, struct nes_cm_listener *); struct nes_cm_node * (*connect)(struct nes_cm_core *, - struct nes_vnic *, struct ietf_mpa_frame *, + struct nes_vnic *, u16, void *, struct nes_cm_info *); int (*close)(struct nes_cm_core *, struct nes_cm_node *); int (*accept)(struct nes_cm_core *, struct ietf_mpa_frame *, struct nes_cm_node *); int (*reject)(struct nes_cm_core *, struct ietf_mpa_frame *, struct nes_cm_node *); - int (*recv_pkt)(struct nes_cm_core *, struct nes_vnic *, + void (*recv_pkt)(struct nes_cm_core *, struct nes_vnic *, struct sk_buff *); int (*destroy_cm_core)(struct nes_cm_core *); int (*get)(struct nes_cm_core *); diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c index 85f26d1..1513d40 100644 --- a/drivers/infiniband/hw/nes/nes_hw.c +++ b/drivers/infiniband/hw/nes/nes_hw.c @@ -2814,7 +2814,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, nesqp = *((struct nes_qp **)&context); if (atomic_inc_return(&nesqp->close_timer_started) == 1) { nesqp->cm_id->add_ref(nesqp->cm_id); - nes_add_ref(&nesqp->ibqp); schedule_nes_timer(nesqp->cm_node, (struct sk_buff *)nesqp, NES_TIMER_TYPE_CLOSE, 1, 0); nes_debug(NES_DBG_AEQ, "QP%u Not decrementing QP refcount (%d)," @@ -2838,7 +2837,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, if (async_event_id == NES_AEQE_AEID_RESET_SENT) { tcp_state = NES_AEQE_TCP_STATE_CLOSED; } - nes_add_ref(&nesqp->ibqp); spin_lock_irqsave(&nesqp->lock, flags); nesqp->hw_iwarp_state = iwarp_state; nesqp->hw_tcp_state = tcp_state; @@ -2876,7 +2874,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, } spin_unlock_irqrestore(&nesqp->lock, flags); if (next_iwarp_state) { - nes_add_ref(&nesqp->ibqp); nes_debug(NES_DBG_AEQ, "issuing hw modifyqp for QP%u. next state = 0x%08X," " also added another reference\n", nesqp->hwqp.qp_id, next_iwarp_state); @@ -2888,7 +2885,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, /* FIN Received but ib state not RTS, close complete will be on its way */ spin_unlock_irqrestore(&nesqp->lock, flags); - nes_rem_ref(&nesqp->ibqp); return; } spin_unlock_irqrestore(&nesqp->lock, flags); @@ -2922,7 +2918,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, if ((tcp_state == NES_AEQE_TCP_STATE_CLOSE_WAIT) || ((nesqp->ibqp_state == IB_QPS_RTS)&& (async_event_id == NES_AEQE_AEID_LLP_CONNECTION_RESET))) { - nes_add_ref(&nesqp->ibqp); nes_cm_disconn(nesqp); } else { nesqp->in_disconnect = 0; @@ -2931,7 +2926,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, break; case NES_AEQE_AEID_LLP_TOO_MANY_RETRIES: nesqp = *((struct nes_qp **)&context); - nes_add_ref(&nesqp->ibqp); spin_lock_irqsave(&nesqp->lock, flags); nesqp->hw_iwarp_state = NES_AEQE_IWARP_STATE_ERROR; nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; @@ -3042,7 +3036,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, nesqp->ibqp.event_handler(&ibevent, nesqp->ibqp.qp_context); } /* tell cm to disconnect, cm will queue work to thread */ - nes_add_ref(&nesqp->ibqp); nes_cm_disconn(nesqp); break; case NES_AEQE_AEID_DDP_UBE_INVALID_MSN_NO_BUFFER_AVAILABLE: @@ -3062,7 +3055,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, nesqp->ibqp.event_handler(&ibevent, nesqp->ibqp.qp_context); } /* tell cm to disconnect, cm will queue work to thread */ - nes_add_ref(&nesqp->ibqp); nes_cm_disconn(nesqp); break; case NES_AEQE_AEID_LLP_RECEIVED_MPA_CRC_ERROR: @@ -3082,7 +3074,6 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev, nesqp->ibqp.event_handler(&ibevent, nesqp->ibqp.qp_context); } /* tell cm to disconnect, cm will queue work to thread */ - nes_add_ref(&nesqp->ibqp); nes_cm_disconn(nesqp); break; /* TODO: additional AEs need to be here */ diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index e3939d1..d79942e 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -2867,7 +2867,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id, attr->qp_state, nesqp->ibqp_state, nesqp->iwarp_state, atomic_read(&nesqp->refcount)); - nes_add_ref(&nesqp->ibqp); spin_lock_irqsave(&nesqp->lock, qplockflags); nes_debug(NES_DBG_MOD_QP, "QP%u: hw_iwarp_state=0x%X, hw_tcp_state=0x%X," @@ -2882,7 +2881,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id); if (nesqp->iwarp_state > (u32)NES_CQP_QP_IWARP_STATE_IDLE) { spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } next_iwarp_state = NES_CQP_QP_IWARP_STATE_IDLE; @@ -2893,7 +2891,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id); if (nesqp->iwarp_state>(u32)NES_CQP_QP_IWARP_STATE_IDLE) { spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } next_iwarp_state = NES_CQP_QP_IWARP_STATE_IDLE; @@ -2904,14 +2901,12 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id); if (nesqp->iwarp_state>(u32)NES_CQP_QP_IWARP_STATE_RTS) { spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } if (nesqp->cm_id == NULL) { nes_debug(NES_DBG_MOD_QP, "QP%u: Failing attempt to move QP to RTS without a CM_ID. \n", nesqp->hwqp.qp_id ); spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } next_iwarp_state = NES_CQP_QP_IWARP_STATE_RTS; @@ -2929,7 +2924,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id, nesqp->hwqp.sq_head, nesqp->hwqp.sq_tail); if (nesqp->iwarp_state == (u32)NES_CQP_QP_IWARP_STATE_CLOSING) { spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return 0; } else { if (nesqp->iwarp_state > (u32)NES_CQP_QP_IWARP_STATE_CLOSING) { @@ -2937,7 +2931,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, " ignored due to current iWARP state\n", nesqp->hwqp.qp_id); spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } if (nesqp->hw_iwarp_state != NES_AEQE_IWARP_STATE_RTS) { @@ -2969,7 +2962,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id); if (nesqp->iwarp_state>=(u32)NES_CQP_QP_IWARP_STATE_TERMINATE) { spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } /* next_iwarp_state = (NES_CQP_QP_IWARP_STATE_TERMINATE | 0x02000000); */ @@ -2982,7 +2974,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, case IB_QPS_RESET: if (nesqp->iwarp_state == (u32)NES_CQP_QP_IWARP_STATE_ERROR) { spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; } nes_debug(NES_DBG_MOD_QP, "QP%u: new state = error\n", @@ -3008,7 +2999,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, break; default: spin_unlock_irqrestore(&nesqp->lock, qplockflags); - nes_rem_ref(&nesqp->ibqp); return -EINVAL; break; } @@ -3088,7 +3078,6 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, nesqp->hwqp.qp_id, atomic_read(&nesqp->refcount), original_last_aeq, nesqp->last_aeq); /* this one is for the cm_disconnect thread */ - nes_add_ref(&nesqp->ibqp); spin_lock_irqsave(&nesqp->lock, qplockflags); nesqp->hw_tcp_state = NES_AEQE_TCP_STATE_CLOSED; nesqp->last_aeq = NES_AEQE_AEID_RESET_SENT; @@ -3097,14 +3086,12 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, } else { nes_debug(NES_DBG_MOD_QP, "QP%u No fake disconnect, QP refcount=%d\n", nesqp->hwqp.qp_id, atomic_read(&nesqp->refcount)); - nes_rem_ref(&nesqp->ibqp); } } else { spin_lock_irqsave(&nesqp->lock, qplockflags); if (nesqp->cm_id) { /* These two are for the timer thread */ if (atomic_inc_return(&nesqp->close_timer_started) == 1) { - nes_add_ref(&nesqp->ibqp); nesqp->cm_id->add_ref(nesqp->cm_id); nes_debug(NES_DBG_MOD_QP, "QP%u Not decrementing QP refcount (%d)," " need ae to finish up, original_last_aeq = 0x%04X." @@ -3128,14 +3115,12 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, " original_last_aeq = 0x%04X. last_aeq = 0x%04X.\n", nesqp->hwqp.qp_id, atomic_read(&nesqp->refcount), original_last_aeq, nesqp->last_aeq); - nes_rem_ref(&nesqp->ibqp); } } else { nes_debug(NES_DBG_MOD_QP, "QP%u Decrementing QP refcount (%d), No ae to finish up," " original_last_aeq = 0x%04X. last_aeq = 0x%04X.\n", nesqp->hwqp.qp_id, atomic_read(&nesqp->refcount), original_last_aeq, nesqp->last_aeq); - nes_rem_ref(&nesqp->ibqp); } err = 0; From rdreier at cisco.com Wed Jul 23 14:27:31 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 23 Jul 2008 14:27:31 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Tue, 22 Jul 2008 23:01:06 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > What's in my mind was to use a common approach for both nonSRQ and SRQ. > Because nonSRQ doesn't have last WQE, but we could do post_send last WR > when receiving DREQ to handle QP resource destroy for both nonSRQ and > SRQ without setting QP to error status. It would be nice if we could make the SRQ and non-SRQ code paths more similar and shrink the code. But I'm not sure there's any reason to post a send work request in the non-SRQ case -- we can just destroy the QP after the last pending receive completes (but we have to wait for them all to complete before destroying the QP). - R. From sashak at voltaire.com Wed Jul 23 14:30:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 00:30:39 +0300 Subject: [ofa-general] reloading/applying configuration changes by opensm In-Reply-To: References: Message-ID: <20080723213039.GE10706@sashak.voltaire.com> Hi, On 14:58 Mon 21 Jul , Hal Rosenstock wrote: > > Sasha has made some changes in this area to make things more dynamic > in terms of rereading all the various config files but this work is in > progress AFAIK. Yes this is exact status. Currently only QoS (high and low levels), partitions and prefix routes configurations will be reloaded. > LMC might work on the master or even that latest tar > ball version (OpenSM 3.2.2). No LMC yet. Sasha From nrocenin_1996 at casadoc.org Wed Jul 23 14:37:54 2008 From: nrocenin_1996 at casadoc.org (Chandrasekar) Date: Wed, 23 Jul 2008 17:37:54 -0400 Subject: [ofa-general] [audio] Church Group Offers Homosexual New Life In Closet Message-ID: Madonnas Former Home Destroyed By Jesus http://hansart.gmxhome.de/stream.html -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From mashirle at us.ibm.com Wed Jul 23 14:57:19 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Wed, 23 Jul 2008 14:57:19 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> On Wed, 2008-07-23 at 14:27 -0700, Roland Dreier wrote: > > What's in my mind was to use a common approach for both nonSRQ and SRQ. > > Because nonSRQ doesn't have last WQE, but we could do post_send last WR > > when receiving DREQ to handle QP resource destroy for both nonSRQ and > > SRQ without setting QP to error status. > > It would be nice if we could make the SRQ and non-SRQ code paths more > similar and shrink the code. But I'm not sure there's any reason to > post a send work request in the non-SRQ case -- we can just destroy the > QP after the last pending receive completes (but we have to wait for > them all to complete before destroying the QP). The reason is non-SRQ shares completion queue just as SRQ. Without post a send work request, we don't know when the QP's last pending recieve completes in the shared completion queue. We might be able to get it but it would induce performance penalty. Thanks Shirley From jlgreath at us.ibm.com Wed Jul 23 15:34:17 2008 From: jlgreath at us.ibm.com (Joseph L Greathouse) Date: Wed, 23 Jul 2008 17:34:17 -0500 Subject: [ofa-general] libibmad-1.6.1:_dump_fields() incorrect for big endian Message-ID: Hi there, libibmad-1.1.6/src/dump.c:_dump_fields() does not always work correctly for big endian machines. Example: 'smpquery portinfo' to a Mellanox ConnectX from a big endian machine prints a number of values incorrectly as zero. (e.g. RespTimeVal:.....................0, when RespTimeVal is actually 16) >From what I can tell, this is an issue between how _get_field() is used in mad_decode_field and how mad_dump_field uses functions like mad_dump_uint........ mad.c:mad_decode_field() sets *(uint32_t *)val = _get_field(), while _get_field() always returns a host-endian uint32_t of the value of the field. This means that on little endian machines, val[0] contains the value of a byte-length field. On big endian machines, val[3] contains this value. mad_dump_field calls, for example, dump.c:mad_dump_uint(). This will check the size (in rounded-up bytes) of the field. For a single byte, it will print *(uint8_t *)val, which is val[0]. This works on little endian, but prints zero on big endian. Possible solution: In the mad_dump_* functions, for cases 1, 2, 3, and 4 in the switch statement, always print *(uint32_t *)val. I believe the dump functions are always fed by _get_field (or get_field64), and thus for values of size 1-4 bytes, any unused bytes will always be zeroed out by _get_field(). I don't want to go around writing patches or filing bug reports all willy-nilly, so I thought it best to just report this and see if what I've come up with makes sense. :) -Joe From rdreier at cisco.com Wed Jul 23 16:17:59 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 23 Jul 2008 16:17:59 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Wed, 23 Jul 2008 14:57:19 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > The reason is non-SRQ shares completion queue just as SRQ. Without post > a send work request, we don't know when the QP's last pending recieve > completes in the shared completion queue. We might be able to get it but > it would induce performance penalty. The current code keeps track of how many receives have been posted to the non-shared RQ, and waits until the last one has completed. I don't see how posting a send request could help you tell when the last receive has completed anyway. The send queue is asynchronous to the receive queue, and so you would have to wait for the last receive to complete -- which means you must have some other way of telling when the last receive completes. - R. From sashak at voltaire.com Wed Jul 23 16:36:17 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 02:36:17 +0300 Subject: [ofa-general] OpenSM "Dead end on path to LID" -- problem with updn, fixed with minhop In-Reply-To: <488762ED.1010307@noaa.gov> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> <4880C61A.6010000@noaa.gov> <48811106.9030906@dev.mellanox.co.il> <4886278B.2080502@noaa.gov> <48871609.5020403@dev.mellanox.co.il> <488762ED.1010307@noaa.gov> Message-ID: <20080723233617.GK10706@sashak.voltaire.com> Hi Nathan, On 10:57 Wed 23 Jul , Nathan Dauchy wrote: > > Given the network topology I described previously, how is "minhop" > expected to behave differently than "updn"? Up/Down guarantees credit loops free routing. In some topologies it drop some connection which "violates" the rule, it may depend on how root switches were detected, etc.. > Unless you recommend otherwise, I now hope to add back in the missing > features and make sure everything is still OK: > LMC=2 > MAXSMPS=4 > TIMEOUT=600 > 12X links on subtree C. > > > > > > Also, where does the opensm run? > > > > For the record, the SM host is connected to one of the "Subtree B" edge > switches. Hardware is an 8-core Intel Xeon 5450 @ 3.00GHz with 16GB > RAM, and a single-port DDR "MT25204" HCA. Software is CentOS-5.1, > linux-2.6.18-53.1.21.el5, and OFED-1.3.1. > > > Please let me know if there is additional diagnostic information I can > gather in order to create a test case for future improvements to the > "updn" routing engine. Could you send us output of ibnetdiscover? And indicate GUID of host where OpenSM is running? Also full OpenSM log could be useful, likely "unknown remote side" problem was unrelated to what you saw, but anyway it would be good to look at how it was resolved. Sasha From sashak at voltaire.com Wed Jul 23 16:55:38 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 02:55:38 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080723235538.GL10706@sashak.voltaire.com> Hi Hal, On 05:42 Thu 10 Jul , Hal Rosenstock wrote: > > Also, what about backward compatibility with apps built with the > previous version of these libraries ? What sort of compatibility problem can we expect? Sasha From sashak at voltaire.com Wed Jul 23 16:58:41 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 02:58:41 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <200807141515.11717.cap@nsc.liu.se> References: <20080709050303.GE12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <200807141515.11717.cap@nsc.liu.se> Message-ID: <20080723235841.GM10706@sashak.voltaire.com> Hi Peter, On 15:15 Mon 14 Jul , Peter Kjellstrom wrote: > > Am I understanding this correctly, you posted tarball.x.y.z.tar.gz and at a > later time modified it but left the old name? If so please don't. No, it is exactly what I didn't want to do. > It's very > nasty to handle situations where there are a bad _and_ a fixed version of the > same file. Fully agree with this. Sasha From mashirle at us.ibm.com Wed Jul 23 17:24:15 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Wed, 23 Jul 2008 17:24:15 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, > The current code keeps track of how many receives have been posted to > the non-shared RQ, and waits until the last one has completed. > > I don't see how posting a send request could help you tell when the last > receive has completed anyway. The send queue is asynchronous to the > receive queue, and so you would have to wait for the last receive to > complete -- which means you must have some other way of telling when the > last receive completes. Do you mean to use p->recv_count to monitor last receive to complete? p->recv_count is always equal to recv_queue_size if post_recv() succeeds. When waitting for p->recv_count to be 0 to release QP, which means we need to either generates unsuccessful CQEs for recv_queue_size times or force ipoib_cm_post_receive_nonsrq() to be failed that many times after DREQ. Do I unstand it correctly? To use post_send last WR is much simple, just wait for this CQE to be generated in CQ then release the resource. The wait time here might be much smaller than above approach. Thanks Shirley From hal.rosenstock at gmail.com Wed Jul 23 19:19:32 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 23 Jul 2008 22:19:32 -0400 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080723235538.GL10706@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> Message-ID: Sasha, On Wed, Jul 23, 2008 at 7:55 PM, Sasha Khapyorsky wrote: > Hi Hal, > > On 05:42 Thu 10 Jul , Hal Rosenstock wrote: >> >> Also, what about backward compatibility with apps built with the >> previous version of these libraries ? > > What sort of compatibility problem can we expect? I'm not entirely sure but the rules have been broken here for libibmad as some APIs changed but the version wasn't updated. Are you saying it's going to work with those changes if the APIs are used ? -- Hal > Sasha > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > From john.savage2 at mail.dcu.ie Wed Jul 23 19:22:04 2008 From: john.savage2 at mail.dcu.ie (Irish Lottery News Center) Date: Wed, 23 Jul 2008 19:22:04 -0700 Subject: [ofa-general] =?iso-8859-1?q?You_have_won_=A3750=2C000GBP?= Message-ID: <4874ECF500012FFE@hawk.dcu.ie> Ref:UK/9420X2/68 Batch:074/05/ZY369 You have won £750, 000.00. To claim your prize,Contact Mr Derek White with the following informations. Name,Address,Age,Occupation,Country Email:irn.support.unit at live.com Mr Clark Johnson From sashak at voltaire.com Wed Jul 23 19:32:52 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 05:32:52 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: References: <20080709050303.GE12551@sashak.voltaire.com> <1215629877.18371.114.camel@hrosenstock-ws.xsigo.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> Message-ID: <20080724023252.GB11924@sashak.voltaire.com> On 22:19 Wed 23 Jul , Hal Rosenstock wrote: > > Are you saying > it's going to work with those changes if the APIs are used ? func(void *p) was changed to func(const void *p), why not? Sasha From hal.rosenstock at gmail.com Wed Jul 23 20:04:09 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 23 Jul 2008 23:04:09 -0400 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080724023252.GB11924@sashak.voltaire.com> References: <20080709050303.GE12551@sashak.voltaire.com> <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> <20080724023252.GB11924@sashak.voltaire.com> Message-ID: On Wed, Jul 23, 2008 at 10:32 PM, Sasha Khapyorsky wrote: > On 22:19 Wed 23 Jul , Hal Rosenstock wrote: >> >> Are you saying >> it's going to work with those changes if the APIs are used ? > > func(void *p) was changed to func(const void *p), why not? Not that one; we already discussed that. I thought there was some other change. Am I mistaken ? -- Hal > > Sasha > From hal.rosenstock at gmail.com Wed Jul 23 20:06:04 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 23 Jul 2008 23:06:04 -0400 Subject: ***SPAM*** Re: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: References: <48849F59.7060502@Voltaire.COM> <48874CED.5020307@gmail.com> <48875C96.70006@gmail.com> Message-ID: On Wed, Jul 23, 2008 at 1:13 PM, Olga Shern (Voltaire) wrote: > In addition to that, we have found more issues with this approach: > > According to IPoIB RFC if SEND ONLY join is used then; > When multicast group doesn't exist then 2 things should be done: > 1.Send multicast packets on all routers multicast group > 2.Register to get multicast group creation / deletion notification > traps from SM > > Therefore the disadvantage of this approach is: > A) Traps mechanism is not scalable. > B) It is complicated mechanism: If multicast group doesn't exist need > to register to get trap from SM on group creation, meanwhile sending > traffic on all routers group. Got trap from SM that multicast have > been created, need to join it and also register for the trap of > multicast group deletion. Start sending traffic on the original > multicast group. If got trap that the group is deleted, need to start > sending traffic on the all routers group …. > > Garbage collector approach is much simpler and more scalable. > > Olga > > P.S. Registration to group creation/deletion notification trap is not > implemented in SA, the change that should be done is also big and > complicated Do you mean SA client rather than SA ? -- Hal > > > On 7/23/08, Yossi Etigin wrote: >> AFAIK OpenSM does not really support send-only, but it uses NonMember to >> emulate it. >> >> Roland Dreier wrote: >> > > The problem worse because ipoib is not really using send-only joins, it >> > > joins as full member. See >> ipoib_multicast.c/ipoib_mcast_sendonly_join(): >> > > > struct ib_sa_mcmember_rec rec = { >> > > #if 0 /* Some SMs don't support >> send-only yet */ >> > > .join_state = 4 >> > > #else >> > > .join_state = 1 >> > > #endif >> > > }; >> > >> > Sorry, you're right. I misread that code. >> > >> > I guess the time has come to use send-only joins... I think all relevant >> > SMs have supported it for a long time. >> > >> > - R. >> > >> > >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From sashak at voltaire.com Wed Jul 23 20:13:56 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 06:13:56 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: References: <20080709190538.GO12551@sashak.voltaire.com> <1215634941.18371.118.camel@hrosenstock-ws.xsigo.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> <20080724023252.GB11924@sashak.voltaire.com> Message-ID: <20080724031356.GD11924@sashak.voltaire.com> On 23:04 Wed 23 Jul , Hal Rosenstock wrote: > On Wed, Jul 23, 2008 at 10:32 PM, Sasha Khapyorsky wrote: > > On 22:19 Wed 23 Jul , Hal Rosenstock wrote: > >> > >> Are you saying > >> it's going to work with those changes if the APIs are used ? > > > > func(void *p) was changed to func(const void *p), why not? > > Not that one; we already discussed that. I thought there was some > other change. Am I mistaken ? So what? new functions? How this will affect backward compatibility? Anyway I see this issue as pretty minor. It is fixed now and will appear in next tarball release. Sasha From sashak at voltaire.com Wed Jul 23 20:25:17 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 06:25:17 +0300 Subject: [ofa-general] [PATCH V2] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080723080339.73f3914f.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> Message-ID: <20080724032517.GE11924@sashak.voltaire.com> Hi Ira, On 08:03 Wed 23 Jul , Ira Weiny wrote: > > Signed-off-by: Ira K. Weiny > --- > opensm/include/iba/ib_types.h | 43 +++++++++++++++++++++++++++++++++++++++++ > opensm/opensm/osm_inform.c | 3 +- > opensm/opensm/osm_trap_rcv.c | 10 +++++++- > 3 files changed, 53 insertions(+), 3 deletions(-) This adds almost 2k text to opensm executable: sashak at sashak ~/src/m/opensm $ size opensm/.libs/old-opensm text data bss dec hex filename 561868 1536 19668 583072 8e5a0 opensm/.libs/opensm sashak at sashak ~/src/m/opensm $ size opensm/.libs/opensm text data bss dec hex filename 563740 1536 19668 584944 8ecf0 opensm/.libs/opensm Wouldn't it be better to put ib_trap_str() to osm_helper.c and not make it inline? Sasha From sashak at voltaire.com Wed Jul 23 21:00:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 07:00:32 +0300 Subject: [ofa-general] Re: [PATCH v2] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop In-Reply-To: <20080710174407.5ab1e4f2.weiny2@llnl.gov> References: <20080710174407.5ab1e4f2.weiny2@llnl.gov> Message-ID: <20080724040032.GF11924@sashak.voltaire.com> On 17:44 Thu 10 Jul , Ira Weiny wrote: > From fa67a47678e9127f5253b909b1a561fdccaf6609 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 1 Jul 2008 14:31:00 -0700 > Subject: [PATCH] infiniband-diags/src/saquery.c: convert GID prints to use inet_ntop > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Wed Jul 23 21:03:36 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 07:03:36 +0300 Subject: [ofa-general] Re: [PATCH v2] infiniband-diags/src/ibaddr.c: convert GID prints to use inet_ntop In-Reply-To: <20080710174408.10a73317.weiny2@llnl.gov> References: <20080710174408.10a73317.weiny2@llnl.gov> Message-ID: <20080724040336.GG11924@sashak.voltaire.com> On 17:44 Thu 10 Jul , Ira Weiny wrote: > From a799f458b9368e99894a38ebf7f88bbbf4057fc9 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 1 Jul 2008 15:48:54 -0700 > Subject: [PATCH] infiniband-diags/src/ibaddr.c: convert GID prints to use inet_ntop > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From ogerlitz at voltaire.com Wed Jul 23 23:18:07 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 24 Jul 2008 09:18:07 +0300 Subject: [ofa-general] Re: [PATCH v2 3/4] rdma/cma: implement RDMA_CM_EVENT_ADDR_CHANGE notification In-Reply-To: References: <48872723.80609@voltaire.com> Message-ID: <48881E9F.204@voltaire.com> Roland Dreier wrote: > I may have forgotten to push. It should be in for-next at least. > Yes, I see now that it indeed in the for-next branch Or. From amirv at mellanox.co.il Wed Jul 23 23:52:44 2008 From: amirv at mellanox.co.il (Amir Vadai) Date: Thu, 24 Jul 2008 09:52:44 +0300 Subject: [ofa-general] [PATCH 1/1] SDP: Don't allow destruct socket when having sdp_destroy_work in workqueue Message-ID: <1216882364527-git-send-email-amirv@mellanox.co.il> in error flow, sdp_destroy_work is placed in workqueue and sometimes the user application call close() that destruct the socket before sdp_destroy_work is called. Signed-off-by: Amir Vadai --- drivers/infiniband/ulp/sdp/sdp_main.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/sdp/sdp_main.c b/drivers/infiniband/ulp/sdp/sdp_main.c index c3bd9f4..de14ab6 100644 --- a/drivers/infiniband/ulp/sdp/sdp_main.c +++ b/drivers/infiniband/ulp/sdp/sdp_main.c @@ -344,6 +344,8 @@ void sdp_reset_sk(struct sock *sk, int rc) sk->sk_state_change(sk); + /* Don't destroy socket before destroy work does its job */ + sock_hold(sk); queue_work(sdp_workqueue, &ssk->destroy_work); read_unlock(&device_removal_lock); @@ -855,6 +857,7 @@ void sdp_destroy_work(struct work_struct *work) but if a CM connection is dropped below our legs state could be any state */ sdp_exch_state(sk, ~0, TCP_CLOSE); + sock_put(sk); } void sdp_dreq_wait_timeout_work(struct work_struct *work) -- 1.5.3 From Robert at saq.co.uk Thu Jul 24 01:05:07 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Thu, 24 Jul 2008 09:05:07 +0100 Subject: [ofa-general] Direct / "Crossover" IPOIB Message-ID: I've just setup two machines linked by a single Inifiniband cable. The IPOIB side of things looks OK, ifconfig shows the interface and each machine can ping itself on its Infiniband IP. They cannot ping each other though and ibstat looks wrong. What do I need to do to make this setup work? Thanks, Rob Ibstat: [root at mrtickle ~]# ibstat CA 'mthca0' CA type: MT25204 Number of ports: 1 Firmware version: 1.2.0 Hardware version: a0 Node GUID: 0x0002c9020022d428 System image GUID: 0x0002c9020022d42b Port 1: State: Initializing Physical state: LinkUp Rate: 20 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 0x02510a68 Port GUID: 0x0002c9020022d429 The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronli.voltaire at gmail.com Thu Jul 24 01:06:24 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Thu, 24 Jul 2008 11:06:24 +0300 Subject: [ofa-general] [PATCH 3/3 v2] ib/uverbs: add support for create_qp_expanded in uverbs In-Reply-To: <200807220959.44327.jackm@dev.mellanox.co.il> References: <200807220959.44327.jackm@dev.mellanox.co.il> Message-ID: <3b5e77ad0807240106t66683547rd29b5dc9b79f0122@mail.gmail.com> > You're still not using a separate cmd structure for ib_uverbs_create_qp_expanded. > It is NOT a good idea to use the 8-bit reserved field for ib_uverbs_create_qp_expanded(). > You should use a different struct (as I indicated in my previous mail). Sorry, I couldn't find the email you were talking about. can you please resend it to me? Why isn't it good to use the reserved field? Will it not just create code duplication for creating a regular qp, and qp_expanded? Ron On Tue, Jul 22, 2008 at 9:59 AM, Jack Morgenstein wrote: > General comment: > Please run checkpatch.pl -- there are lots of formatting errors here. > See additional comments below. > > - Jack > > On Monday 21 July 2008 23:27, ronli at voltaire.com wrote: >> -ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, >> - const char __user *buf, int in_len, >> - int out_len) >> +static ssize_t ib_uverbs_create_qp_common(struct ib_uverbs_file *file, >> + const char __user *buf, int in_len, >> + int out_len, int expanded) >> { > >> - struct ib_uverbs_create_qp cmd; >> + struct ib_uverbs_create_qp cmd; > unneeded modification > >> struct ib_uverbs_create_qp_resp resp; >> struct ib_udata udata; >> struct ib_uqp_object *obj; >> @@ -1078,7 +1078,6 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, >> goto err_put; >> } >> >> - attr.create_flags = 0; >> attr.event_handler = ib_uverbs_qp_event_handler; >> attr.qp_context = file; >> attr.send_cq = scq; >> @@ -1087,7 +1086,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, >> attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; >> attr.qp_type = cmd.qp_type; >> attr.xrc_domain = xrcd; >> - attr.create_flags = 0; >> + attr.create_flags = expanded ? cmd.reserved : 0; > You're still not using a separate cmd structure for ib_uverbs_create_qp_expanded. > It is NOT a good idea to use the 8-bit reserved field for ib_uverbs_create_qp_expanded(). > You should use a different struct (as I indicated in my previous mail). > > The common part is more difficult to isolate, since you have different cmd structs, but > it is possible. > >> >> @@ -722,6 +726,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, >> return ERR_PTR(-ENOMEM); >> >> err = create_qp_common(dev, pd, init_attr, udata, 0, qp); >> + > unneeded change >> if (err) { >> kfree(qp); >> return ERR_PTR(err); > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From ronli.voltaire at gmail.com Thu Jul 24 01:07:55 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Thu, 24 Jul 2008 11:07:55 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 1/3 v2] libibverbs : add new verbs: create_qp_expanded In-Reply-To: <200807221009.30350.jackm@dev.mellanox.co.il> References: <200807221009.30350.jackm@dev.mellanox.co.il> Message-ID: <3b5e77ad0807240107m4bffcf3dsdaef3a4c1eb80173@mail.gmail.com> should be: __u8 reserved; __u32 create_flags; __u32 reserved1; Why do we need a 32 bit create_flags if the uverbs layer has only 8 bits? Ron On Tue, Jul 22, 2008 at 10:09 AM, Jack Morgenstein wrote: > On Monday 21 July 2008 23:25, ronli at voltaire.com wrote: >> +struct ibv_create_qp_expanded { >> + __u32 command; >> + __u16 in_words; >> + __u16 out_words; >> + __u64 response; >> + __u64 user_handle; >> + __u32 pd_handle; >> + __u32 send_cq_handle; >> + __u32 recv_cq_handle; >> + __u32 srq_handle; >> + __u32 max_send_wr; >> + __u32 max_recv_wr; >> + __u32 max_send_sge; >> + __u32 max_recv_sge; >> + __u32 max_inline_data; >> + __u8 sq_sig_all; >> + __u8 qp_type; >> + __u8 is_srq; > > >> + __u8 create_flags; > should be: > __u8 reserved; > __u32 create_flags; > __u32 reserved1; > > The reserved1 field is needed for the proper alignment of the u64 driver_data field > in all situations. > >> + __u64 driver_data[0]; >> +}; >> + > > - Jack > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From jackm at dev.mellanox.co.il Thu Jul 24 01:11:48 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 24 Jul 2008 11:11:48 +0300 Subject: [ofa-general] ***SPAM*** [PATCH 1/3 v2] libibverbs : add new verbs: create_qp_expanded In-Reply-To: <3b5e77ad0807240107m4bffcf3dsdaef3a4c1eb80173@mail.gmail.com> References: <200807221009.30350.jackm@dev.mellanox.co.il> <3b5e77ad0807240107m4bffcf3dsdaef3a4c1eb80173@mail.gmail.com> Message-ID: <200807241111.48990.jackm@dev.mellanox.co.il> On Thursday 24 July 2008 11:07, Ron Livne wrote: > should be: > __u8 reserved; > __u32 create_flags; > __u32 reserved1; > > Why do we need a 32 bit create_flags if the uverbs layer has only 8 bits? > > Ron The uverbs layer also should have 32 bits. These fields (libibverbs and uverbs) need to be coordinated. 8 bits is not enough -- we'll soon run out of bits at the rate bits are being added to the create-flags field. - Jack > > > On Tue, Jul 22, 2008 at 10:09 AM, Jack Morgenstein > wrote: > > On Monday 21 July 2008 23:25, ronli at voltaire.com wrote: > >> +struct ibv_create_qp_expanded { > >> + __u32 command; > >> + __u16 in_words; > >> + __u16 out_words; > >> + __u64 response; > >> + __u64 user_handle; > >> + __u32 pd_handle; > >> + __u32 send_cq_handle; > >> + __u32 recv_cq_handle; > >> + __u32 srq_handle; > >> + __u32 max_send_wr; > >> + __u32 max_recv_wr; > >> + __u32 max_send_sge; > >> + __u32 max_recv_sge; > >> + __u32 max_inline_data; > >> + __u8 sq_sig_all; > >> + __u8 qp_type; > >> + __u8 is_srq; > > > > > >> + __u8 create_flags; > > should be: > > __u8 reserved; > > __u32 create_flags; > > __u32 reserved1; > > > > The reserved1 field is needed for the proper alignment of the u64 driver_data field > > in all situations. > > > >> + __u64 driver_data[0]; > >> +}; > >> + > > > > - Jack > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > From jackm at dev.mellanox.co.il Thu Jul 24 01:30:26 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 24 Jul 2008 11:30:26 +0300 Subject: [ofa-general] [PATCH 3/3 v2] ib/uverbs: add support for create_qp_expanded in uverbs In-Reply-To: <3b5e77ad0807240106t66683547rd29b5dc9b79f0122@mail.gmail.com> References: <200807220959.44327.jackm@dev.mellanox.co.il> <3b5e77ad0807240106t66683547rd29b5dc9b79f0122@mail.gmail.com> Message-ID: <200807241130.26668.jackm@dev.mellanox.co.il> On Thursday 24 July 2008 11:06, Ron Livne wrote: > > You're still not using a separate cmd structure for ib_uverbs_create_qp_expanded. > > It is NOT a good idea to use the 8-bit reserved field for ib_uverbs_create_qp_expanded(). > > You should use a different struct (as I indicated in my previous mail). > > Sorry, I couldn't find the email you were talking about. can you > please resend it to me? > Why isn't it good to use the reserved field? Will it not just create > code duplication for creating a regular qp, and qp_expanded? It will create code duplication -- this is what I was referring to when I suggested that you write a helper function in my e-mail of July 21 entitled "Re: [ofa-general] [PATCH 3/3] ib/uverbs: add support for create_qp_expanded in uverbs". When I wrote the following in my feedback for libibverbs of July 21 entitled "Re: [ofa-general] [PATCH 1/3] libibverbs : add new verbs: create_qp_expanded": You should not need to touch struct ibv_create_qp You should define struct ibv_create_qp_expanded { ... __u8 sq_sig_all; __u8 qp_type; __u8 is_srq; __u8 reserved; __u32 create_flags; __u32 reserved2; __u64 driver_data[0]; }; and use that all along the way. I meant that you should continue this into kernel space, and create a corresponding uverbs structure: struct ib_uverbs_create_qp_expanded { __u64 response; __u64 user_handle; __u32 pd_handle; __u32 send_cq_handle; __u32 recv_cq_handle; __u32 srq_handle; __u32 max_send_wr; __u32 max_recv_wr; __u32 max_send_sge; __u32 max_recv_sge; __u32 max_inline_data; __u8 sq_sig_all; __u8 qp_type; __u8 is_srq; __u8 reserved; __u32 create_flags; __u32 reserved2; __u64 driver_data[0]; }; and use this in the ib_uverbs_create_qp_expanded function. You can filter out the common elements in ib_uverbs_create_qp_expanded() and ib_uverbs_create_qp() and write an ib_uverbs_create_qp_common() function so as not to duplicate things. - Jack From ogerlitz at voltaire.com Thu Jul 24 01:44:28 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 24 Jul 2008 11:44:28 +0300 Subject: [ofa-general] Direct / "Crossover" IPOIB In-Reply-To: References: Message-ID: <488840EC.1020207@voltaire.com> Robert Dunkley wrote: > > I’ve just setup two machines linked by a single Inifiniband cable. The > IPOIB side of things looks OK, ifconfig shows the interface and each > machine can ping itself on its Infiniband IP. They cannot ping each > other though and ibstat looks wrong. What do I need to do to make this > setup work? > > Port 1: > > State: Initializing > > Physical state: LinkUp > > Base lid: 0 > > SM lid: 0 > you have to run SM (Subnet Manager), if you issue $ opensm on one of the nodes, it will do that. Or. From nkune1970 at smt98.ro Thu Jul 24 02:11:26 2008 From: nkune1970 at smt98.ro (Sonja) Date: Thu, 24 Jul 2008 11:11:26 +0200 Subject: [ofa-general] Female politician caught in wild scandal Message-ID: <000a01c8ed6d$406b6f60$c7ff74c0@ORLYPC> We have evidence of your crimes watch now -------------- next part -------------- An HTML attachment was scrubbed... URL: From Robert at saq.co.uk Thu Jul 24 02:31:35 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Thu, 24 Jul 2008 10:31:35 +0100 Subject: [ofa-general] Direct / "Crossover" IPOIB References: <488840EC.1020207@voltaire.com> Message-ID: Thanks Or, I have installed and run opensm on one of the machines, I can now move traffic between them. Pings show a latency og 0.07-0.09ms which seems OK. Qperf shows abysmal results in connected mode though: [root at mrhappy OFED-1.3.1]# qperf 192.168.10.161 tcp_bw tcp_lat tcp_bw: bw = 32.8 KB/sec tcp_lat: latency = 42.2 us Datagram mode shows better bandwidth but still awful latency: [root at mrtickle OFED-1.3.1]# qperf 192.168.10.160 tcp_bw tcp_lat tcp_bw: bw = 400 MB/sec tcp_lat: latency = 41.2 us Is there something wrong with my setup? (QPerf seems to think so) Thanks, Rob -----Original Message----- From: Or Gerlitz [mailto:ogerlitz at voltaire.com] Sent: 24 July 2008 09:44 To: Robert Dunkley Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] Direct / "Crossover" IPOIB Robert Dunkley wrote: > > I've just setup two machines linked by a single Inifiniband cable. The > IPOIB side of things looks OK, ifconfig shows the interface and each > machine can ping itself on its Infiniband IP. They cannot ping each > other though and ibstat looks wrong. What do I need to do to make this > setup work? > > Port 1: > > State: Initializing > > Physical state: LinkUp > > Base lid: 0 > > SM lid: 0 > you have to run SM (Subnet Manager), if you issue $ opensm on one of the nodes, it will do that. Or. The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield From vlad at lists.openfabrics.org Thu Jul 24 02:54:31 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Thu, 24 Jul 2008 02:54:31 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080724-0200 daily build status Message-ID: <20080724095432.2904EE60D52@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080724-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080724-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080724-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080724-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080724-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080724-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.25 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' Makefile:514: /home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.25/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.25' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.26 Log: -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ ' \ modules make[1]: Entering directory `/home/vlad/kernel.org/ppc64/linux-2.6.26' Makefile:518: /home/vlad/kernel.org/ppc64/linux-2.6.26/arch/ppc64/Makefile: No such file or directory make[1]: *** No rule to make target `/home/vlad/kernel.org/ppc64/linux-2.6.26/arch/ppc64/Makefile'. Stop. make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.26' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From Robert at saq.co.uk Thu Jul 24 04:40:31 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Thu, 24 Jul 2008 12:40:31 +0100 Subject: [ofa-general] Direct / "Crossover" IPOIB References: <488840EC.1020207@voltaire.com> Message-ID: Just a general question and I apologise if my questions seem simplistic. I have played with the larger MTU settings in connected mode (32768 seems to work well) and now have some much better results. What kind of bandwidth do other users see with TCP on qperf? I will try with different PCI latencies set in the bios but are there any other tweaks I should look at? Config: Tyan S4989 Quad Opteron Opteron 8350 16Gb Ram Mellanox 20Gb PCIE-X8 card Direct Connect to identical system Centos 5.2 & OFED 1.3.1 Results: [root at mrtickle device]# qperf 192.168.10.160 tcp_bw tcp_lat tcp_bw: bw = 671 MB/sec tcp_lat: latency = 31.1 us [root at mrtickle device]# qperf 192.168.10.160 rc_bi_bw rc_bi_bw: bw = 1.71 GB/sec -----Original Message----- From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Robert Dunkley Sent: 24 July 2008 10:32 To: Or Gerlitz Cc: general at lists.openfabrics.org Subject: RE: [ofa-general] Direct / "Crossover" IPOIB Thanks Or, I have installed and run opensm on one of the machines, I can now move traffic between them. Pings show a latency og 0.07-0.09ms which seems OK. Qperf shows abysmal results in connected mode though: [root at mrhappy OFED-1.3.1]# qperf 192.168.10.161 tcp_bw tcp_lat tcp_bw: bw = 32.8 KB/sec tcp_lat: latency = 42.2 us Datagram mode shows better bandwidth but still awful latency: [root at mrtickle OFED-1.3.1]# qperf 192.168.10.160 tcp_bw tcp_lat tcp_bw: bw = 400 MB/sec tcp_lat: latency = 41.2 us Is there something wrong with my setup? (QPerf seems to think so) Thanks, Rob -----Original Message----- From: Or Gerlitz [mailto:ogerlitz at voltaire.com] Sent: 24 July 2008 09:44 To: Robert Dunkley Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] Direct / "Crossover" IPOIB Robert Dunkley wrote: > > I've just setup two machines linked by a single Inifiniband cable. The > IPOIB side of things looks OK, ifconfig shows the interface and each > machine can ping itself on its Infiniband IP. They cannot ping each > other though and ibstat looks wrong. What do I need to do to make this > setup work? > > Port 1: > > State: Initializing > > Physical state: LinkUp > > Base lid: 0 > > SM lid: 0 > you have to run SM (Subnet Manager), if you issue $ opensm on one of the nodes, it will do that. Or. The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From 54ddcrawuw at asi-solutions.com Thu Jul 24 03:01:09 2008 From: 54ddcrawuw at asi-solutions.com (dal salah) Date: Thu, 24 Jul 2008 10:01:09 +0000 Subject: [ofa-general] Best of Chanel, Dior, Versace Message-ID: <000701c8ed83$04f32350$31b9c9bf@txsbeab> The world's generous de luxe store for shoes and bags is just one click away. Recommended by tens thousands of grateful customers worldwide, we carry many of famous brands including: Gucci Adidas Chanel Shoes Dior UGG Here you willc see millions of best designs for shoes, and leather products, at at cheapest priceE. Sale close this week, so visit us now and start pampering yourself and your loved ones! - Visit our site: www.steertheirs[DOT]com (copy this link and replace "[DOT]" to ".") From olga.shern at gmail.com Thu Jul 24 06:15:06 2008 From: olga.shern at gmail.com (Olga Shern (Voltaire)) Date: Thu, 24 Jul 2008 16:15:06 +0300 Subject: ***SPAM*** Re: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: References: <48849F59.7060502@Voltaire.COM> <48874CED.5020307@gmail.com> <48875C96.70006@gmail.com> Message-ID: On 7/24/08, Hal Rosenstock wrote: > On Wed, Jul 23, 2008 at 1:13 PM, Olga Shern (Voltaire) > wrote: > > In addition to that, we have found more issues with this approach: > > > > According to IPoIB RFC if SEND ONLY join is used then; > > When multicast group doesn't exist then 2 things should be done: > > 1.Send multicast packets on all routers multicast group > > 2.Register to get multicast group creation / deletion notification > > traps from SM > > > > Therefore the disadvantage of this approach is: > > A) Traps mechanism is not scalable. > > B) It is complicated mechanism: If multicast group doesn't exist need > > to register to get trap from SM on group creation, meanwhile sending > > traffic on all routers group. Got trap from SM that multicast have > > been created, need to join it and also register for the trap of > > multicast group deletion. Start sending traffic on the original > > multicast group. If got trap that the group is deleted, need to start > > sending traffic on the all routers group …. > > > > Garbage collector approach is much simpler and more scalable. > > > > Olga > > > > P.S. Registration to group creation/deletion notification trap is not > > implemented in SA, the change that should be done is also big and > > complicated > > Do you mean SA client rather than SA ? > yes > -- Hal > > > > > > > On 7/23/08, Yossi Etigin wrote: > >> AFAIK OpenSM does not really support send-only, but it uses NonMember to > >> emulate it. > >> > >> Roland Dreier wrote: > >> > > The problem worse because ipoib is not really using send-only joins, it > >> > > joins as full member. See > >> ipoib_multicast.c/ipoib_mcast_sendonly_join(): > >> > > > struct ib_sa_mcmember_rec rec = { > >> > > #if 0 /* Some SMs don't support > >> send-only yet */ > >> > > .join_state = 4 > >> > > #else > >> > > .join_state = 1 > >> > > #endif > >> > > }; > >> > > >> > Sorry, you're right. I misread that code. > >> > > >> > I guess the time has come to use send-only joins... I think all relevant > >> > SMs have supported it for a long time. > >> > > >> > - R. > >> > > >> > > >> _______________________________________________ > >> general mailing list > >> general at lists.openfabrics.org > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit > >> http://openib.org/mailman/listinfo/openib-general > >> > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > From kliteyn at dev.mellanox.co.il Thu Jul 24 06:48:46 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 24 Jul 2008 16:48:46 +0300 Subject: [ofa-general] Direct / "Crossover" IPOIB In-Reply-To: References: <488840EC.1020207@voltaire.com> Message-ID: <4888883E.2040109@dev.mellanox.co.il> Robert, Not directly answering your question, but just some general info: Robert Dunkley wrote: > Config: > Tyan S4989 Quad Opteron > Opteron 8350 > 16Gb Ram > Mellanox 20Gb PCIE-X8 card > Direct Connect to identical system > Centos 5.2 & OFED 1.3.1 > > Results: > [root at mrtickle device]# qperf 192.168.10.160 tcp_bw tcp_lat > tcp_bw: > bw = 671 MB/sec Don't know what *exactly* is the max bw for tcp_bw (to me the number looks ok), but note that the bottleneck here is IPoIB. You have a quad-core CPU, so you can run four parallel tests, and their combined bw would be similar to RDMA traffic bw, which is 1.71 GB/s in your case. > tcp_lat: > latency = 31.1 us > > [root at mrtickle device]# qperf 192.168.10.160 rc_bi_bw > rc_bi_bw: > bw = 1.71 GB/sec This one looks fine (I guess you have PCI Express Gen2, right?). -- Yevgeny > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Robert > Dunkley > Sent: 24 July 2008 10:32 > To: Or Gerlitz > Cc: general at lists.openfabrics.org > Subject: RE: [ofa-general] Direct / "Crossover" IPOIB > > Thanks Or, > > I have installed and run opensm on one of the machines, I can now move > traffic between them. > > Pings show a latency og 0.07-0.09ms which seems OK. > > Qperf shows abysmal results in connected mode though: > [root at mrhappy OFED-1.3.1]# qperf 192.168.10.161 tcp_bw tcp_lat > tcp_bw: > bw = 32.8 KB/sec > tcp_lat: > latency = 42.2 us > > Datagram mode shows better bandwidth but still awful latency: > [root at mrtickle OFED-1.3.1]# qperf 192.168.10.160 tcp_bw tcp_lat > tcp_bw: > bw = 400 MB/sec > tcp_lat: > latency = 41.2 us > > Is there something wrong with my setup? (QPerf seems to think so) > > Thanks, > > Rob > > -----Original Message----- > From: Or Gerlitz [mailto:ogerlitz at voltaire.com] > Sent: 24 July 2008 09:44 > To: Robert Dunkley > Cc: general at lists.openfabrics.org > Subject: Re: [ofa-general] Direct / "Crossover" IPOIB > > Robert Dunkley wrote: >> I've just setup two machines linked by a single Inifiniband cable. The > >> IPOIB side of things looks OK, ifconfig shows the interface and each >> machine can ping itself on its Infiniband IP. They cannot ping each >> other though and ibstat looks wrong. What do I need to do to make this > >> setup work? >> >> Port 1: >> >> State: Initializing >> >> Physical state: LinkUp >> >> Base lid: 0 >> >> SM lid: 0 >> > you have to run SM (Subnet Manager), if you issue > > $ opensm > > on one of the nodes, it will do that. > > Or. > > > The SAQ Group > Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ > SEMTEC Limited trading as SAQ is Registered in England & Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > SAQ Group Delivers high quality, honestly priced communication and I.T. > services to UK Business. > DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : > Backups : Managed Networks : Remote Support. > > Find us in http://www.thebestof.co.uk/petersfield > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From ronli.voltaire at gmail.com Thu Jul 24 08:04:01 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Thu, 24 Jul 2008 18:04:01 +0300 Subject: [ofa-general] Re: [PATCH 1/3 v2] libibverbs : add new verbs: create_qp_expanded In-Reply-To: References: Message-ID: <3b5e77ad0807240804x230c564frbe0dbdc22b189c10@mail.gmail.com> > + return ibv_cmd_create_qp_expanded(pd, qp, attr, 0, cmd, cmd_size, resp, resp_size); Roland, you are right. This is not a good solution for old-kernel/new-libibverbs. It seems that I will have to create a bit messier code (extracting common code in 2 functions into 1 function) in order to avoid the compatibility issue. Maybe you thought of a better way? Ron On Tue, Jul 22, 2008 at 8:21 AM, Roland Dreier wrote: > > --- a/src/cmd.c > > +++ b/src/cmd.c > > @@ -625,8 +625,18 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, > > struct ibv_create_qp *cmd, size_t cmd_size, > > struct ibv_create_qp_resp *resp, size_t resp_size) > > { > > - IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP, resp, resp_size); > > + return ibv_cmd_create_qp_expanded(pd, qp, attr, 0, cmd, cmd_size, resp, resp_size); > > +} > > Guys, please try to be careful about compatibility... we care about both > new-kernel/old-libibverbs and old-kernel/new-libibverbs. In this case > think about what happens if we merge this into libibverbs and then a > user tries that code on an old kernel. > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From rdreier at cisco.com Thu Jul 24 08:41:04 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 24 Jul 2008 08:41:04 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get the some more RDMA/InfiniBand changes for 2.6.27, all of which have been in linux-next: Amir Vadai (1): RDMA/cma: Add RDMA_CM_EVENT_TIMEWAIT_EXIT event Dotan Barak (1): RDMA/iwcm: Remove IB_ACCESS_LOCAL_WRITE from remote QP attributes Jack Morgenstein (2): mlx4_core: Add module parameter to enable QoS support mlx4_code: Add missing FW status return code Joachim Fenkes (2): IB/ehca: Filter PATH_MIG events if QP was never armed IB/ehca: Use default value for Local CA ACK Delay if FW returns 0 Julia Lawall (1): IB/ehca: Release mutex in error path of alloc_small_queue_page() Or Gerlitz (3): RDMA/cma: Add RDMA_CM_EVENT_ADDR_CHANGE event IB/iser: Add support for RDMA_CM_EVENT_ADDR_CHANGE event IPoIB: Include err code in trace message for ib_sa_path_rec_get() failures Ralph Campbell (1): IB/sa_query: Check if sm_ah is NULL in ib_sa_remove_one() Roland Dreier (7): IB/mlx4: Rename struct mlx4_lso_seg to mlx4_wqe_lso_seg mlx4_core: Keep free count for MTT buddy allocator IB/mthca: Keep free count for MTT buddy allocator IB/mlx4: Add support for memory management extensions and local DMA L_Key mlx4_core: Improve error message when not enough UAR pages are available MAINTAINERS: Remove Glenn Streiff from NetEffect entry Merge branches 'bkl-removal', 'cma', 'ehca', 'for-2.6.27', 'mlx4', 'mthca' and 'nes' into for-linus MAINTAINERS | 2 - drivers/infiniband/core/cma.c | 99 ++++++++++++++++++++++++++++- drivers/infiniband/core/iwcm.c | 3 +- drivers/infiniband/core/sa_query.c | 3 +- drivers/infiniband/hw/ehca/ehca_classes.h | 1 + drivers/infiniband/hw/ehca/ehca_hca.c | 4 +- drivers/infiniband/hw/ehca/ehca_irq.c | 4 + drivers/infiniband/hw/ehca/ehca_qp.c | 2 + drivers/infiniband/hw/ehca/ipz_pt_fn.c | 1 + drivers/infiniband/hw/mlx4/cq.c | 12 ++++ drivers/infiniband/hw/mlx4/main.c | 11 +++ drivers/infiniband/hw/mlx4/mlx4_ib.h | 15 +++++ drivers/infiniband/hw/mlx4/mr.c | 70 ++++++++++++++++++++ drivers/infiniband/hw/mlx4/qp.c | 74 ++++++++++++++++++++-- drivers/infiniband/hw/mthca/mthca_dev.h | 1 + drivers/infiniband/hw/mthca/mthca_mr.c | 26 +++++--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 +- drivers/infiniband/ulp/iser/iser_verbs.c | 1 + drivers/net/mlx4/cmd.c | 3 + drivers/net/mlx4/fw.c | 18 +++-- drivers/net/mlx4/fw.h | 2 +- drivers/net/mlx4/main.c | 2 + drivers/net/mlx4/mlx4.h | 1 + drivers/net/mlx4/mr.c | 49 +++++++++++---- drivers/net/mlx4/pd.c | 7 ++ include/linux/mlx4/device.h | 10 +++ include/linux/mlx4/qp.h | 18 ++++-- include/rdma/rdma_cm.h | 4 +- 28 files changed, 396 insertions(+), 49 deletions(-) From weiny2 at llnl.gov Thu Jul 24 09:02:47 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 24 Jul 2008 09:02:47 -0700 Subject: [ofa-general] [PATCH V2] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080724032517.GE11924@sashak.voltaire.com> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> Message-ID: <20080724090247.64d8c0e6.weiny2@llnl.gov> I just followed the convention used but yea I think you are right. Ira On Thu, 24 Jul 2008 06:25:17 +0300 Sasha Khapyorsky wrote: > Hi Ira, > > On 08:03 Wed 23 Jul , Ira Weiny wrote: > > > > Signed-off-by: Ira K. Weiny > > --- > > opensm/include/iba/ib_types.h | 43 +++++++++++++++++++++++++++++++++++++++++ > > opensm/opensm/osm_inform.c | 3 +- > > opensm/opensm/osm_trap_rcv.c | 10 +++++++- > > 3 files changed, 53 insertions(+), 3 deletions(-) > > This adds almost 2k text to opensm executable: > > sashak at sashak ~/src/m/opensm $ size opensm/.libs/old-opensm > text data bss dec hex filename > 561868 1536 19668 583072 8e5a0 opensm/.libs/opensm > sashak at sashak ~/src/m/opensm $ size opensm/.libs/opensm > text data bss dec hex filename > 563740 1536 19668 584944 8ecf0 opensm/.libs/opensm > > Wouldn't it be better to put ib_trap_str() to osm_helper.c and not make > it inline? > > Sasha From declan.quinn4 at mail.dcu.ie Thu Jul 24 09:10:14 2008 From: declan.quinn4 at mail.dcu.ie (The National Lottery) Date: Thu, 24 Jul 2008 09:10:14 -0700 Subject: [ofa-general] Prize Winning Email Message-ID: <4874ECF50001544D@hawk.dcu.ie> The Camelot Group,: Operators of The National Lottery. 3b Olympic Way, Sefton Business Park, Aintree, Liverpool , L30 1RD WINNING NOTIFICATION The United Kingdom National Lottery wishes to inform you that the results of the E-mail address ballot lottery international program by Great Britain 1306 held on Saturday 28th of June 2008, as part of weekly promotions bonus. Your mail account have been picked as a winner of a lump sum pay out of Eight hundred and ninty-one thousand, nine hundred and thirty-four Great Britain pounds (£891,934.00 pounds sterlings) in cash credited to file REF NO. REF:UKL/74-A0802742008. This is from total prize money of GBP £4,459,670.00 shared among the FIVE (5) international winners in this category. To begin your claim, please contact your Claims Agent: Mr. Sean James. Email:cg.claimsdesk at live.com Tel:+447031944691 Fax:+44870 478 2022 FUNDS RELEASE FORM 1.FULL NAMES:________2.ADDRESS:__________City________State_________Postcode___ ___Country 3.TELEPHONE NUMBER:__________ 4.AGE:________5.OCCUPATION: Note below your winning details: Serial Nº: A069-07 Lotto Lucky No:07 12 19 23 46 48 Bonus 28 File REF Nº.: UKL/74-A0802742008 BATCH Nº.: 2008UKL-01 From yevgenyp at mellanox.co.il Thu Jul 24 09:27:02 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Thu, 24 Jul 2008 19:27:02 +0300 Subject: [ofa-general][PATCH] mlx4_core: Add Vlan tag field to control segment Message-ID: <4888AD56.8000004@mellanox.co.il> Hello Roland, This fix is according to ConnectX PRM and required for Ethernet driver. >From bcb7beea7733b6b13bada8bc4009dabd8a833131 Mon Sep 17 00:00:00 2001 From: Yevgeny Petrilin Date: Wed, 23 Jul 2008 11:44:56 +0300 Subject: [PATCH] mlx4_core: Vlan tag in wqe control segment. Added fields for vlan tag an a flag for insert Vlan to the packet. Those fields are in use for Ethernet packets. Signed-off-by: Yevgeny Petrilin --- include/linux/mlx4/qp.h | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index f02e9ed..5ab9c3e 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -168,7 +168,9 @@ enum { struct mlx4_wqe_ctrl_seg { __be32 owner_opcode; - u8 reserved2[3]; + __be16 vlan_tag; +#define MLX4_WQE_CTRL_INS_VLAN 0x40 + u8 ins_vlan; u8 fence_size; /* * High 24 bits are SRC remote buffer; low 8 bits are flags: -- 1.5.3.7 From sashak at voltaire.com Thu Jul 24 09:45:21 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 19:45:21 +0300 Subject: [ofa-general] Re: [PATCH][TRIVIAL] opensm/osm_lin_fwd_tbl.h: Cosmetic changes In-Reply-To: <1215695118.18371.157.camel@hrosenstock-ws.xsigo.com> References: <1215695118.18371.157.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080724164521.GB14872@sashak.voltaire.com> On 06:05 Thu 10 Jul , Hal Rosenstock wrote: > opensm/osm_lin_fwd_tbl.h: Cosmetic changes > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Thu Jul 24 09:45:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 19:45:39 +0300 Subject: [ofa-general] Re: [PATCH][TRIVIAL] opensm/osm_rand_fwd_tbl.h: Cosmetic changes In-Reply-To: <1215695844.18371.160.camel@hrosenstock-ws.xsigo.com> References: <1215695844.18371.160.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080724164539.GC14872@sashak.voltaire.com> On 06:17 Thu 10 Jul , Hal Rosenstock wrote: > opensm/osm_rand_fwd_tbl.h: Cosmetic changes > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Thu Jul 24 09:46:28 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 19:46:28 +0300 Subject: [ofa-general] Re: [PATCH] OpenSM: convert GID prints to use inet_ntop In-Reply-To: <20080710174410.1261bc4f.weiny2@llnl.gov> References: <20080710174410.1261bc4f.weiny2@llnl.gov> Message-ID: <20080724164628.GD14872@sashak.voltaire.com> On 17:44 Thu 10 Jul , Ira Weiny wrote: > From 5638ddc23276e0d83179d01fb0ee7bcd735f4ab0 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 10 Jul 2008 17:35:37 -0700 > Subject: [PATCH] OpenSM: convert GID prints to use inet_ntop > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha. From hal.rosenstock at gmail.com Thu Jul 24 09:54:02 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 24 Jul 2008 12:54:02 -0400 Subject: [ofa-general] ***SPAM*** [PATCH] opensm/osm_console_io.c: Eliminate compile warnings Message-ID: opensm/osm_console_io.c: Eliminate compile warnings osm_console_io.c: In function `osm_console_init': osm_console_io.c:164: warning: implicit declaration of function `socket' osm_console_io.c:170: warning: implicit declaration of function `setsockopt' osm_console_io.c:178: warning: implicit declaration of function `bind' osm_console_io.c:184: warning: implicit declaration of function `listen' Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/osm_console_io.c b/opensm/opensm/osm_console_io.c index ee0b632..b46ab7b 100644 --- a/opensm/opensm/osm_console_io.c +++ b/opensm/opensm/osm_console_io.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2005-2007 Voltaire, Inc. All rights reserved. + * Copyright (c) 2008 HNR Consulting. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU @@ -49,6 +50,7 @@ #include #include #include +#include #endif #include #include From hal.rosenstock at gmail.com Thu Jul 24 09:54:10 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 24 Jul 2008 12:54:10 -0400 Subject: [ofa-general] ***SPAM*** [PATCH] opensm/osm_qos_parser.y: Eliminate bison warning Message-ID: opensm/osm_qos_parser.y: Eliminate bison warning bison -y -d -o ./osm_qos_parser_y.c -p__qos_parser_ ./osm_qos_parser.y ./osm_qos_parser.y:414: warning: previous rule lacks an ending `;' Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y index ed588a8..6fa024c 100644 --- a/opensm/opensm/osm_qos_parser.y +++ b/opensm/opensm/osm_qos_parser.y @@ -3,6 +3,7 @@ * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * Copyright (c) 2008 HNR Consulting. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU @@ -410,6 +411,7 @@ qos_setup_items: /* empty */ /* Parsing vlarb-tables */ vlarb_tables: TK_VLARB_TABLES_START vlarb_scope_items TK_VLARB_TABLES_END + ; vlarb_scope_items: /* empty */ | vlarb_scope_items vlarb_scope From weiny2 at llnl.gov Thu Jul 24 10:11:13 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 24 Jul 2008 10:11:13 -0700 Subject: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080724090247.64d8c0e6.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> Message-ID: <20080724101113.76537e5e.weiny2@llnl.gov> >From 1f810dd6ae9a73549cacdbb26f464e6132675592 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 22 Jul 2008 16:31:36 -0700 Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 2 + opensm/opensm/libopensm.map | 1 + opensm/opensm/osm_helper.c | 44 +++++++++++++++++++++++++++++++++++++++++ opensm/opensm/osm_inform.c | 3 +- opensm/opensm/osm_trap_rcv.c | 10 +++++++- 5 files changed, 57 insertions(+), 3 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 09ec257..fffce00 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7219,6 +7219,8 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) +char * OSM_API ib_get_trap_str(uint16_t trap_num); + /****f* IBA Base: Types/ib_notice_is_generic * NAME * ib_notice_is_generic diff --git a/opensm/opensm/libopensm.map b/opensm/opensm/libopensm.map index 759a4e8..7cd2aba 100644 --- a/opensm/opensm/libopensm.map +++ b/opensm/opensm/libopensm.map @@ -17,6 +17,7 @@ OPENSM_1.5 { ib_get_sm_method_str; ib_get_sm_attr_str; ib_get_sa_attr_str; + ib_get_trap_str; osm_dump_port_info; osm_dump_portinfo_record; osm_dump_guidinfo_record; diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index 21ff51c..cc76d5c 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -2338,3 +2338,47 @@ const char *osm_get_sm_mgr_state_str(IN uint16_t state) __osm_sm_mgr_state_str[state] : __osm_sm_mgr_state_str[ARR_SIZE(__osm_sm_mgr_state_str) - 1]; } + +char * OSM_API +ib_get_trap_str(uint16_t trap_num) +{ + switch(trap_num) + { + case 64: + return ("GID in service"); + case 65: + return ("GID out of service"); + case 66: + return ("New mcast group created"); + case 67: + return ("Mcast group deleted"); + case 68: + return ("UnPath, Path no longer valid"); + case 69: + return ("RePath, Path recomputed"); + case 128: + return ("Link state change"); + case 129: + return ("Local Link integrity threshold reached"); + case 130: + return ("Excessive Buffer Overrun Threshold reached"); + case 131: + return ("Flow Control Update watchdog timer expired"); + case 144: + return ("CapabilityMask, NodeDescription, Link [Width|Speed] Enabled changed"); + case 145: + return ("System Image GUID changed"); + case 256: + return ("Bad M_Key"); + case 257: + return ("Bad P_Key"); + case 258: + return ("Bad Q_Key"); + case 259: + return ("Bad P_Key (switch external port)"); + default: + break; + } + return ("Unknown"); +} + diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index e6146f7..ebc13bb 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, /* an official Event information log */ if (ib_notice_is_generic(p_ntc)) { OSM_LOG(p_log, OSM_LOG_INFO, - "Reporting Generic Notice type:%u num:%u" + "Reporting Generic Notice type:%u num:%u (%s)" " from LID:%u GID:0x%016" PRIx64 ",0x%016" PRIx64 "\n", ib_notice_get_type(p_ntc), cl_ntoh16(p_ntc->g_or_v.generic.trap_num), + ib_get_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), cl_ntoh16(p_ntc->issuer_lid), cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index ae7ab05..03cbd26 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, || (p_ntci->g_or_v.generic.trap_num == CL_HTON16(131))) OSM_LOG(sm->p_log, OSM_LOG_ERROR, - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " + "Received Generic Notice type:0x%02X " + "num:%u (%s) Producer:%u (%s) " "from LID:%u Port %d TID:0x%016" PRIx64 "\n", ib_notice_get_type(p_ntci), cl_ntoh16(p_ntci->g_or_v.generic. trap_num), + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. + trap_num)), cl_ntoh32(ib_notice_get_prod_type (p_ntci)), ib_get_producer_type_str @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, ); else OSM_LOG(sm->p_log, OSM_LOG_ERROR, - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " + "Received Generic Notice type:0x%02X " + "num:%u (%s) Producer:%u (%s) " "from LID:%u TID:0x%016" PRIx64 "\n", ib_notice_get_type(p_ntci), cl_ntoh16(p_ntci->g_or_v.generic. trap_num), + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. + trap_num)), cl_ntoh32(ib_notice_get_prod_type (p_ntci)), ib_get_producer_type_str -- 1.5.4.5 From sashak at voltaire.com Thu Jul 24 10:26:25 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 20:26:25 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_console_io.c: Eliminate compile warnings In-Reply-To: References: Message-ID: <20080724172625.GE14872@sashak.voltaire.com> On 12:54 Thu 24 Jul , Hal Rosenstock wrote: > opensm/osm_console_io.c: Eliminate compile warnings > > osm_console_io.c: In function `osm_console_init': > osm_console_io.c:164: warning: implicit declaration of function `socket' > osm_console_io.c:170: warning: implicit declaration of function `setsockopt' > osm_console_io.c:178: warning: implicit declaration of function `bind' > osm_console_io.c:184: warning: implicit declaration of function `listen' > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Thu Jul 24 10:26:47 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 20:26:47 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_parser.y: Eliminate bison warning In-Reply-To: References: Message-ID: <20080724172647.GF14872@sashak.voltaire.com> On 12:54 Thu 24 Jul , Hal Rosenstock wrote: > opensm/osm_qos_parser.y: Eliminate bison warning > > bison -y -d -o ./osm_qos_parser_y.c -p__qos_parser_ ./osm_qos_parser.y > ./osm_qos_parser.y:414: warning: previous rule lacks an ending `;' > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Thu Jul 24 10:53:37 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 20:53:37 +0300 Subject: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080724101113.76537e5e.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> <20080724101113.76537e5e.weiny2@llnl.gov> Message-ID: <20080724175337.GG14872@sashak.voltaire.com> On 10:11 Thu 24 Jul , Ira Weiny wrote: > From 1f810dd6ae9a73549cacdbb26f464e6132675592 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 22 Jul 2008 16:31:36 -0700 > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM > > > Signed-off-by: Ira K. Weiny Applied with minor changes - similar to ib_get_sa_attr_str() and ib_get_sa_attr_str() I moved ib_get_trap_str() prototype to osm_helper.h and made trap_num parameter in network byte order. Thanks! Sasha From sashak at voltaire.com Thu Jul 24 11:06:48 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 21:06:48 +0300 Subject: [ofa-general] Issues with osm_lin_fwd_tbl.h:osm_lin_fwd_tbl_get_lids_per_block() In-Reply-To: <1215695111.18371.155.camel@hrosenstock-ws.xsigo.com> References: <4874BF35.4060805@bull.net> <1215695111.18371.155.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080724180648.GH14872@sashak.voltaire.com> Hi, On 06:05 Thu 10 Jul , Hal Rosenstock wrote: > > > Finally, what is the point in keeping the unused parameter p_tbl ? > > Yes, passing that parameter doesn't appear to be needed and could be > eliminated. In fact, that whole routine could be eliminated and replaced > by a define IMO and then have that ripple back into/simplify any > routines which use it. Moving this to macro looks fine for me. Sasha From weiny2 at llnl.gov Thu Jul 24 11:11:49 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 24 Jul 2008 11:11:49 -0700 Subject: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080724175337.GG14872@sashak.voltaire.com> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> <20080724101113.76537e5e.weiny2@llnl.gov> <20080724175337.GG14872@sashak.voltaire.com> Message-ID: <20080724111149.0d59cf4d.weiny2@llnl.gov> On Thu, 24 Jul 2008 20:53:37 +0300 Sasha Khapyorsky wrote: > On 10:11 Thu 24 Jul , Ira Weiny wrote: > > From 1f810dd6ae9a73549cacdbb26f464e6132675592 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Tue, 22 Jul 2008 16:31:36 -0700 > > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM > > > > > > Signed-off-by: Ira K. Weiny > > Applied with minor changes - similar to ib_get_sa_attr_str() and > ib_get_sa_attr_str() I moved ib_get_trap_str() prototype to osm_helper.h > and made trap_num parameter in network byte order. Ah, ok. I specifically made it in host byte order. I figured that would be easier going forward but it is your call. Ira From sashak at voltaire.com Thu Jul 24 11:12:47 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 24 Jul 2008 21:12:47 +0300 Subject: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080724111149.0d59cf4d.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> <20080724101113.76537e5e.weiny2@llnl.gov> <20080724175337.GG14872@sashak.voltaire.com> <20080724111149.0d59cf4d.weiny2@llnl.gov> Message-ID: <20080724181247.GJ14872@sashak.voltaire.com> On 11:11 Thu 24 Jul , Ira Weiny wrote: > > Ah, ok. I specifically made it in host byte order. I figured that would be > easier going forward but it is your call. I'm fine if we will change it later - currently I saw that all its usages require network byte order. Sasha From tziporet at mellanox.co.il Thu Jul 24 12:05:21 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Thu, 24 Jul 2008 22:05:21 +0300 Subject: [ofa-general] OFED 1.4 alpha release is ready Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> Hi, OFED 1.4 alpha release is available on http://www.openfabrics.org/downloads/OFED/ofed-1.4/OFED-1.4-alpha1.tgz To get BUILD_ID run ofed_info Please report any issues in bugzilla https://bugs.openfabrics.org/ for OFED 1.4 The beta release is expected on August 7 2008 Tziporet & Vlad ======================================================================== Release information: -------------------- Linux Operating Systems: - RedHat EL4 up5: 2.6.9-55.ELsmp - RedHat EL4 up6: 2.6.9-67.ELsmp - RedHat EL5: 2.6.18-8.el5 - RedHat EL5 up1: 2.6.18-53.el5 - RedHat EL5 up2: 2.6.18-92.el5 - Fedora C9: 2.6.25-14.fc9 * - SLES10: 2.6.16.21-0.8-smp - SLES10 SP1: 2.6.16.46-0.12-smp - SLES10 SP1 up1: 2.6.16.53-0.16-smp - SLES10 SP2: 2.6.16.60-0.21-smp - OpenSuSE 10.3: 2.6.22.5-31 * - kernel.org: 2.6.26 * OSes that are partially tested Systems: * x86_64 * x86 * ia64 * ppc64 Main Changes from OFED 1.3 ========================== 1. General changes o Kernel code based on 2.6.26 (will be 2.6.27) o Added iSER target package o Added NFS-RDMA support (for 2.6.26 only for now) 2. IPoIB o LRO support - when we move to 2.6.27 3. SDP o Bug fixes in the state machine and close flow 4. qlgc_vnic o Support for hotswap of EVIC and dynamic update of existing connections with the addition of QLogic dynamic update daemon. o Performance improvements in handling of Ethernet broadcast/multicast traffic. 5. RDS (for now only kernel 2.6.26 is working) o GA of RDMA API (using FMRs) - RDS API version 3 o iWARP support 6. uDAPL o Added socket based CM - for both scalability and interop with Windows o Added UD extensions - for version 2.0 only o v1 library package has been renamed to compat-dapl-1.2.8-1 7. Management o OpenSM - APM - disjoint paths - Path balancing for LMC + console diagnostics - OpenSM configuration unification - MGID to MLID mapping for IPv6 SNM - Routing engines chain - IBA 1.2.1 additions - Failover/Handover improvements o ibutils: - Congestion Control - Report created in CSV format o Diagnostic tools: - ibnetdiscover library - to accelerate another tools 8. Low level drivers: o mlx4: Enable Virtual Protocol Interconnect - Eth and IB on the same device Tasks that should be completed for the beta: ============================================ 1. Move to kernel base 2.6.27-rc1 with the features: - New verbs to support BMME (Fast memory thru send queue, Local invalidate send work requests, Read with invalidate. - IPoIB LRO 2. RDS to work on distro OSes 3. NFS-RDMA to work on distro OSes 4. New MPI versions: MVAPICH 1.1, OpenMPI 1.3, MVAPICH2 1.2 5. OSM: Cashed routing From Nathan.Dauchy at noaa.gov Thu Jul 24 17:50:24 2008 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Thu, 24 Jul 2008 18:50:24 -0600 Subject: [ofa-general] OpenSM "Dead end on path to LID" -- problem with pdn, fixed with minhop In-Reply-To: <20080723233617.GK10706@sashak.voltaire.com> References: <487E886B.8010906@noaa.gov> <487EA1CC.7060308@noaa.gov> <4880B181.2090400@dev.mellanox.co.il> <4880C61A.6010000@noaa.gov> <48811106.9030906@dev.mellanox.co.il> <4886278B.2080502@noaa.gov> <48871609.5020403@dev.mellanox.co.il> <488762ED.1010307@noaa.gov> <20080723233617.GK10706@sashak.voltaire.com> Message-ID: <48892350.9050500@noaa.gov> Sasha Khapyorsky wrote: > On 10:57 Wed 23 Jul , Nathan Dauchy wrote: >> >> Please let me know if there is additional diagnostic information I can >> gather in order to create a test case for future improvements to the >> "updn" routing engine. > > Could you send us output of ibnetdiscover? And indicate GUID of host > where OpenSM is running? Sure. I guess I'll send a tarball off-list. If anyone besides Yevgeny and Sasha is interested, please let me know. > Also full OpenSM log could be useful, likely "unknown remote side" > problem was unrelated to what you saw, but anyway it would be good to > look at how it was resolved. I'll try to gather that up too. -Nathan From sfr at canb.auug.org.au Thu Jul 24 20:44:44 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Fri, 25 Jul 2008 13:44:44 +1000 Subject: [ofa-general] linux-next: manual merge of the infiniband tree Message-ID: <20080725134444.9e67772c.sfr@canb.auug.org.au> Hi all, Today's linux-next merge of the infiniband tree got a conflict in include/linux/mlx4/qp.h between commit 95d04f0735b4fc837bff9aedcc3f3efb20ddc3d1 ("IB/mlx4: Add support for memory management extensions and local DMA L_Key") from Linus' tree and commit 5fcbaab14db959eb6feecae57387cfff364f11b3 ("IB/mlx4: Add support for memory management extensions and local DMA L_Key") from the infiniband tree. Please don't submit patches to Linus that are different to those you have in linux-next. I used the version in Linus' tree. -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From rdreier at cisco.com Thu Jul 24 20:50:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 24 Jul 2008 20:50:05 -0700 Subject: [ofa-general] Re: linux-next: manual merge of the infiniband tree In-Reply-To: <20080725134444.9e67772c.sfr@canb.auug.org.au> (Stephen Rothwell's message of "Fri, 25 Jul 2008 13:44:44 +1000") References: <20080725134444.9e67772c.sfr@canb.auug.org.au> Message-ID: > Today's linux-next merge of the infiniband tree got a conflict in > include/linux/mlx4/qp.h between commit > 95d04f0735b4fc837bff9aedcc3f3efb20ddc3d1 ("IB/mlx4: Add support for > memory management extensions and local DMA L_Key") from Linus' tree and > commit 5fcbaab14db959eb6feecae57387cfff364f11b3 ("IB/mlx4: Add support > for memory management extensions and local DMA L_Key") from the > infiniband tree. > > Please don't submit patches to Linus that are different to those you have > in linux-next. Sorry, someone pointed out a bug in the patch just before I sent it to Linus. I've updated my tree to Linus's, so this conflict should be gone tomorrow. Thanks, Roland From sfr at canb.auug.org.au Thu Jul 24 21:36:53 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Fri, 25 Jul 2008 14:36:53 +1000 Subject: [ofa-general] Re: linux-next: manual merge of the infiniband tree In-Reply-To: References: <20080725134444.9e67772c.sfr@canb.auug.org.au> Message-ID: <20080725143653.a2233ac7.sfr@canb.auug.org.au> Hi Roland, On Thu, 24 Jul 2008 20:50:05 -0700 Roland Dreier wrote: > > Sorry, someone pointed out a bug in the patch just before I sent it to > Linus. I've updated my tree to Linus's, so this conflict should be gone > tomorrow. It happens. Thanks. (I am just in "driver the point home" mood :-)) -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From iuzzolin at nmia.com Thu Jul 24 22:39:32 2008 From: iuzzolin at nmia.com (Harold/Carlyn Iuzzolino) Date: Thu, 24 Jul 2008 23:39:32 -0600 Subject: [ofa-general] Problem with ipath_dma.c and dma-mapping.h Message-ID: <200807250539.m6P5dWc04020@gandalf.iuzzolino.com> Dear Openfabrics, general at lists.openfabrics.org I am trying to compile OFED-1.4-20080724-0600.tgz, the latest-and-greatest version as of today July 24. Fedora Core release 6 (Zod) Kernel 2.6.18-1.2798.fc6xen on an x86_64 [root at elrond ~]# uname -a Linux elrond 2.6.18-1.2798.fc6xen #1 SMP Mon Oct 16 14:59:01 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux It is a Single core 64 bit 2.4 GHz machine I run the install.pl script with no changes and made these choices OFED Distribution Software Installation Menu 2) Install OFED Software OFED Distribution Software Installation Menu 3) All packages (all of Basic, HPC) Please choose an implementation of MVAPICH2: 1) OFA (IB and iWARP) Enable ROMIO support [Y/n]: y Enable shared library support [Y/n]: y Enable Checkpoint-Restart support [y/N]: n I have run into the following error: /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c:58: error: implicit declaration of function 'valid_dma_direction' The code it is complaining about is ipath_dma.c: BUG_ON(!valid_dma_direction(direction)); This subroutine valid_dma_direction is USED 6 times in one file ipath_dma.c but isn't DEFINED anywhere. If I cd up 3 levels and search recursively for valid_dma_direction, I find valid_dma_direction in 24 files, all dma-mapping.h for different versions of the kernel_addons backports. dma-mapping.h has this kind of code: #ifndef __x86_64__ /* x86_64 was the only architecture that had valid_dma_direction * before 2.6.18 */ static inline int valid_dma_direction(int dma_direction) { return ((dma_direction == DMA_BIDIRECTIONAL) || (dma_direction == DMA_TO_DEVICE) || (dma_direction == DMA_FROM_DEVICE)); } #endif I added the line #include to the file ipath_dma.c (/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c) That didn't work. The reason is that since this is a 64 bit machine, the definition: static inline int valid_dma_direction(int dma_direction) isn't included because of the #ifndef __x86_64__ statement. So I went into the dma-mapping.h file and put #undef __x86_64__ before the ifndef __x86_64__ so that the definition of valid_dma_direction(int dma_direction) is included. SUCCESS, sort of. That subroutine DID compile. BUT a file later on didn't compile. BUT I shouldn't have to do fixes like that. And the question is, if this is a 64 bit machine, and the dma-mapping.h defines valid_dma_direction ONLY IF you are NOT compiling for a 64 bit machine, then the Openfabrics files: ipath_dma.c and/or dma-mapping.h have some kind of mistake. Has anybody else run into this problem? Do you know what the correct code is? Can you get Openfabrics to incorporate the fix into their code? Carlyn Iuzzolino From Robert at saq.co.uk Fri Jul 25 02:32:58 2008 From: Robert at saq.co.uk (Robert Dunkley) Date: Fri, 25 Jul 2008 10:32:58 +0100 Subject: [ofa-general] Terrible TCP Qperf Performance with Centos 5.2 and default settings Message-ID: I see very low qperf TCP performance of IPOIB on Centos 5.2 by default (RDMA performance is fine). Centos sets the maximum MTU for ib0 on boot automatically. For CM centos sets the maximum MTU that is just slightly less than the logical 65536 setting. Connected mode only seems to like nice round MTU values (eg. 32768 or 2048) so the default setting always gives terrible performance. Datagram mode seems to have no problem with odd MTU values like 2044 (This seems to be the max in Datagram mode). The other thing I noticed was Centos warns that multicast will be broken with MTUs above 2044, this seems to be true, I don't need it so disabled multicast on the Infiniband interface. Is the behaviour I'm seeing normal or is this some kind of bug? (Either in OFED running on Centos 5.2 or QPerf) Results with various settings below. Rob Default (Connected mode) [root at mrhappy OFED-1.3.1]# qperf 192.168.10.161 tcp_bw tcp_lat tcp_bw: bw = 32.8 KB/sec tcp_lat: latency = 42.2 us Datagram Mode with 2044 MTU (Max for Datagram): [root at mrtickle OFED-1.3.1]# qperf 192.168.10.160 tcp_bw tcp_lat tcp_bw: bw = 400 MB/sec tcp_lat: latency = 41.2 us Connected Mode with 32768 MTU: [root at mrtickle ~]# qperf 192.168.10.160 tcp_bw tcp_lat tcp_bw: bw = 866 MB/sec tcp_lat: latency = 30.8 us The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ SEMTEC Limited trading as SAQ is Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. Find us in http://www.thebestof.co.uk/petersfield -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Fri Jul 25 02:52:23 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Fri, 25 Jul 2008 02:52:23 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080725-0200 daily build status Message-ID: <20080725095223.C6A18E60DA2@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.24 Passed on ia64 with linux-2.6.25 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080725-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080725-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080725-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080725-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080725-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080725-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From hal.rosenstock at gmail.com Fri Jul 25 04:08:07 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 25 Jul 2008 07:08:07 -0400 Subject: [ofa-general] Cisco 7008 Internal/External Port Mappings In-Reply-To: References: <20080710123304.GD4004@mtls03> <5388B4A2-14D8-41C2-96D9-19E96ABAD673@hpc.ufl.edu> <1215725788.18371.193.camel@hrosenstock-ws.xsigo.com> Message-ID: Ted, On Fri, Jul 11, 2008 at 12:35 PM, Ted Wilcox (tewilcox) wrote: > Attached is a text file (unix file format) with the port mappings. As > you surmised, the leaf chips are in slots 9, 10, 13, 14, and the spines > are in 11, 12. Our nomenclature is that the leaf cards are "Node" cards > (Node 1 = Slot 9, Node 2 = Slot 10, etc.) and the spine cards are "Core" > cards (Core 1 = slot 11, Core 2 = slot 12). Is this only for TS270/Cisco 7008 ? -- Hal > I apologize for the somewhat goofy naming; it came from the early hardware design. > > If you're running the Cisco SM, we have recently released a pretty > general-purpose network debugging/maintenance tool called FACT which may > help you out > (http://tools.cisco.com/support/downloads/go/Redirect.x?mdfid=279966112 > -- look under "Server Fabric Software"). It requires a cisco.com login, > but not a support contract. > > -Ted. > > P.S. I don't generally keep up with ofa-general (my apologies), so > please cc' me on any replies, flames, etc. > > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Charles > Taylor > Sent: Thursday, July 10, 2008 4:15 PM > To: general-list > Subject: Re: [ofa-general] Cisco 7008 Internal/External Port Mappings > > > Nope. Looks pretty much the same as ibnetdiscover without the "- > g". It would be really useful if it *did* work. :) > > Thanks for the suggestion though. I think we just may be stuck > mapping it empirically. I'm sure the pattern will become obvious > quickly enough. > > Charlie Taylor > UF HPC Center > > On Jul 10, 2008, at 5:36 PM, Hal Rosenstock wrote: > >> On Thu, 2008-07-10 at 17:27 -0400, Charles Taylor wrote: >>> We've got a couple of Cisco 7008/Topspin 270 core switches. Much >>> of >>> the diagnostic and topology information we get from the ib diag tools > >>> in OFED refer to the internal ports rather than the actual external >>> slot/port. >> >> Does ibnetdiscover -g work ? There was some work done to do this >> mapping for Cisco but it may be out of date. >> >> -- Hal >> >>> We could deduce the mappings, I suppose, by tracing cables and >>> using ibtracert between nodes but that seems like a real >>> pain. I was wondering if anyone on this list knows how that >>> mapping >>> works. The chassis itself consists of 8 "ib port" blades in the >>> front labelled as slots 1 - 8. There are then six "fabric >>> controller" slots in the back of the chassis labelled 9 - 14. >>> Slots 11 and 12 are "special" as they hold the "master" and "standby >>> master" fabric controllers. >>> >>> Seems like the FC's in slots 11 and 12 connect the the FCs in slots >>> 9,10,13,14 which in turn connect to the external port blades in slots >>> 1 - 8. >>> >>> Again, all of the diagnostic information refers to the internal FC >>> ports and mapping those to the external ports (to which we actually >>> connect leaf switches) seems non-trivial. I think there was even a >>> document somewhere that described the mapping. We just can't find >>> it >>> now. >>> >>> Thanks, >>> >>> Charlie Taylor >>> UF HPC Center >>> _______________________________________________ >>> general mailing list >>> general at lists.openfabrics.org >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>> >>> To unsubscribe, please visit >>> http://openib.org/mailman/listinfo/openib-general >> > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From hal.rosenstock at gmail.com Fri Jul 25 06:05:52 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 25 Jul 2008 09:05:52 -0400 Subject: [ofa-general] [PATCH] ibsim/sim_mad.c: Cosmetic changes Message-ID: ibsim/sim_mad.c: Cosmetic changes Fix typo in IBWARN message Make attribute ID format consistent (in hex) Also, other cosmetic formatting change Signed-off-by: Hal Rosenstock diff --git a/ibsim/sim_mad.c b/ibsim/sim_mad.c index 35417fd..b8ce2ab 100644 --- a/ibsim/sim_mad.c +++ b/ibsim/sim_mad.c @@ -1130,7 +1130,7 @@ Smpfn *get_handle_fn(ib_rpc_t rpc, int response) return fn; } - return 0; // No MGTCLASS matched . + return 0; // No MGTCLASS matched. } int process_packet(Client * cl, void *p, int size, Client ** dcl) @@ -1182,7 +1182,7 @@ int process_packet(Client * cl, void *p, int size, Client rpc.mgtclass, rpc.attr.id); goto _dropped; } - VERB("forward pkt to client %d pid %d attr %d", + VERB("forward pkt to client %d pid %d attr 0x%x", (*dcl)->id, (*dcl)->pid, rpc.attr.id); forward_MAD(r->mad, &rpc, &path); return sizeof(*r); // forward only @@ -1289,7 +1289,7 @@ int send_trap(Port * port, int trapnum) Port *destport; if (!encode_trapfn) { - IBWARN("trap number %d not upported", trapnum); + IBWARN("trap number %d not supported", trapnum); return -1; } From 31nottinghill at lesliemarsh.co.uk Fri Jul 25 04:29:20 2008 From: 31nottinghill at lesliemarsh.co.uk (averil irving) Date: Fri, 25 Jul 2008 11:29:20 +0000 Subject: [ofa-general] to general Message-ID: <000501c8ee58$06076a30$85473aaf@drrnjv> Get the great discounts on popular software today ! All software is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.profeklore[DOT]com (copy this link and then replace "[DOT]" to ".") From hal.rosenstock at gmail.com Fri Jul 25 06:27:58 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 25 Jul 2008 09:27:58 -0400 Subject: [ofa-general] [PATCH] libibumad/umad.c: Cosmetic change Message-ID: libibumad/umad.c: Cosmetic change Signed-off-by: Hal Rosenstock diff --git a/libibumad/src/umad.c b/libibumad/src/umad.c index dcc7275..64fd38a 100644 --- a/libibumad/src/umad.c +++ b/libibumad/src/umad.c @@ -220,7 +220,7 @@ release_ca(umad_ca_t *ca) * if *port > 0, check ca[port] state. Otherwise set *port to * the first port that is active, and if such is not found, to * the first port that is link up and if none are linkup, then - * the first port that is not disabled. Otherwise return -1; + * the first port that is not disabled. Otherwise return -1. */ static int resolve_ca_port(char *ca_name, int *port) From hal.rosenstock at gmail.com Fri Jul 25 06:27:47 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 25 Jul 2008 09:27:47 -0400 Subject: [ofa-general] ***SPAM*** [PATCH] libibmad/ChangeLog: Fix typo Message-ID: libibmad/ChangeLog: Fix typo Signed-off-by: Hal Rosenstock diff --git a/libibmad/ChangeLog b/libibmad/ChangeLog index bd88ca0..0c4c337 100644 --- a/libibmad/ChangeLog +++ b/libibmad/ChangeLog @@ -1,7 +1,7 @@ 2008-06-15 Max Matveev * Add extra entry points for most functions to allow explicit selection of HCA for MAD rpc. - * constify MAD port pointer for mad_rcp, mad_rpc_rmpp and sa_rpc_call + * Constify MAD port pointer for mad_rpc, mad_rpc_rmpp, and sa_rpc_call 2007-07-10 Hal Rosenstock From AHKumar at odu.edu Fri Jul 25 09:13:25 2008 From: AHKumar at odu.edu (Amit H. Kumar) Date: Fri, 25 Jul 2008 12:13:25 -0400 Subject: [ofa-general] ib-bonding-debuginfo was not created Message-ID: Hello ... Installing OFED 1.3.1 for the first time. I get the following message on building RPMS and then installtion stops. It creates Kernel-ib, kernel-ib-devel and ib-bonding rpms and then fails at the following. Running rpmbuild --rebuild --define '_topdir /var/tmp/OFED_topdir' --define 'KVERSION 2.6.18-53.1.21.el5' --define '_release 2.6.18_53.1.21.el5' --define '_prefix /usr' /share/apps/OFED-1.3.1/SRPMS/ib-bonding-0.9.0-25.src.rpm ib-bonding-debuginfo was not created Any idea what could I be doing wrong ??? Thank you, Amit -------------- next part -------------- An HTML attachment was scrubbed... URL: From qcfjzlvki at surferboard.com Fri Jul 25 10:41:55 2008 From: qcfjzlvki at surferboard.com (rachel) Date: Fri, 25 Jul 2008 09:41:55 -0800 (PDT) Subject: [ofa-general] it`s rachel Message-ID: <479794798920630fdv4nan1cx1.3666053967263.Mail.news@roof.z2c.net> Hi It`s rachel again. Will you ever contact me? I made those nude pictures especially for you and I wont write to you again! If you wanna see them just drop me a line at: brachel11 at wildpears.info From kapilar at wangligroup.com Fri Jul 25 09:13:27 2008 From: kapilar at wangligroup.com (Mortgagee Trust Union 2008 num_919) Date: Fri, 25 Jul 2008 20:13:27 +0400 Subject: [ofa-general] New mortgagee investment program for North America Message-ID: <000c01c8ee79$c130674c$6bfe8155@xp-41a9e1d223b8> Mortgagee Trust Union Represented by Andrew Brown Address: 178 Hurst Street, Birmingham B5 4TB, UK Phone: +44 121 288 0053 This message is of utmost importance to all of you! Mortgagee Trust Union is ready to offer new employment opportunities to responsible individuals. Our Union was founded 6 years ago and ever since trust and joint support of all our members have been at the very heart of our success, financial growth and solvent reputation. Despite global oil and mortgage crisis, we believe that our future lies in the hands of independent investment. Mortgagee Trust Union offers you to become one of our affiliates. It is possible to apply from almost every region of Europe and North America because our investment program already applies to hundreds of independent investors from these regions. Thatù›™s when we need responsible individuals to cooperate with Mortgagee Trust Union processing department. ù›œMonetary operatorù› is the vacancy we are ready to offer to you. Part-time employment with a minimum earning of 2750 USD per month. All applicants must match the requirements below: Be over 20.y.o US Citizenship is not obligatory. Being a resident is enough Regular Internet access, phone connection (home and mobile). If you wish to get more information about Mortgagee Trust Union and current employment opportunities, CLICK HERE to confirm the region you are from Thank You very much for your time With warmest regards Andrew Brown Msg-Id: 36668833 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Fri Jul 25 10:20:48 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 25 Jul 2008 10:20:48 -0700 Subject: [ofa-general] Problem with ipath_dma.c and dma-mapping.h In-Reply-To: <200807250539.m6P5dWc04020@gandalf.iuzzolino.com> (Harold's message of "Thu, 24 Jul 2008 23:39:32 -0600") References: <200807250539.m6P5dWc04020@gandalf.iuzzolino.com> Message-ID: > BUT I shouldn't have to do fixes like that. And the question is, if > this is a 64 bit machine, and the dma-mapping.h defines valid_dma_direction > ONLY IF you are NOT compiling for a 64 bit machine, then the Openfabrics > files: ipath_dma.c and/or dma-mapping.h have some kind of mistake. It's a broken backport. valid_dma_direction() was moved into generic code between 2.6.18 and 2.6.19, so someone needs to add a backport patch to handle kernels 2.6.18 and older. - R. From rdreier at cisco.com Fri Jul 25 10:25:16 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 25 Jul 2008 10:25:16 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Wed, 23 Jul 2008 17:24:15 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > Do you mean to use p->recv_count to monitor last receive to complete? > p->recv_count is always equal to recv_queue_size if post_recv() > succeeds. When waitting for p->recv_count to be 0 to release QP, which > means we need to either generates unsuccessful CQEs for recv_queue_size > times or force ipoib_cm_post_receive_nonsrq() to be failed that many > times after DREQ. Do I unstand it correctly? The current code at least tries to do this: it decrements the recv_count on every receive completion with unsuccessful status, and moves an rx countext to the reap list when its last receive completes. I don't see any obvious bug here; is it not working in your tests? > To use post_send last WR is much simple, just wait for this CQE to be > generated in CQ then release the resource. The wait time here might be > much smaller than above approach. But when do you post the send work request? This only works if you already know the last receive has completed, because there is no synchronization required between the receive queue and the send queue, and so the completion of the send work request doesn't tell you anything you didn't already know. - R. From rdreier at cisco.com Fri Jul 25 10:37:29 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 25 Jul 2008 10:37:29 -0700 Subject: [ofa-general] Re: [PATCH V2 for 2.6.27] mlx4: Update/add Mellanox Technologies copyright lines to mlx4 driver files In-Reply-To: <200807171517.00673.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Thu, 17 Jul 2008 15:17:00 +0300") References: <200807171517.00673.jackm@dev.mellanox.co.il> Message-ID: thanks, applied. > V2: > eliminated copyright additions/changes for files Mellanox did > not modify. please put patch history comments (that shouldn't go into the final kernel history) -after- the "---" line so they get automatically stripped. From rdreier at cisco.com Fri Jul 25 10:38:41 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 25 Jul 2008 10:38:41 -0700 Subject: [ofa-general][PATCH] mlx4_core: Add Vlan tag field to control segment In-Reply-To: <4888AD56.8000004@mellanox.co.il> (Yevgeny Petrilin's message of "Thu, 24 Jul 2008 19:27:02 +0300") References: <4888AD56.8000004@mellanox.co.il> Message-ID: Thanks, applied. > Hello Roland, > > This fix is according to ConnectX PRM and required for Ethernet driver. please put commentary that isn't intended for the final kernel log *after* the "---" line so git tools strip it automatically (rather than forcing me to edit it by hand). > >From bcb7beea7733b6b13bada8bc4009dabd8a833131 Mon Sep 17 00:00:00 2001 > From: Yevgeny Petrilin > Date: Wed, 23 Jul 2008 11:44:56 +0300 > Subject: [PATCH] mlx4_core: Vlan tag in wqe control segment. Please leave this header out of the email (the "From:" line with your author information doesn't hurt, but everything else has to be edited out by hand). - R. From dotanba at gmail.com Fri Jul 25 12:41:38 2008 From: dotanba at gmail.com (Dotan Barak) Date: Fri, 25 Jul 2008 21:41:38 +0200 Subject: ***SPAM*** Re: [ofa-general] Terrible TCP Qperf Performance with Centos 5.2 and default settings In-Reply-To: References: Message-ID: <488A2C72.4000203@gmail.com> Robert Dunkley wrote: > > I see very low qperf TCP performance of IPOIB on Centos 5.2 by default > (RDMA performance is fine). > Which HW do you use IB or IWarp? thanks Dotan > > Centos sets the maximum MTU for ib0 on boot automatically. For CM > centos sets the maximum MTU that is just slightly less than the > logical 65536 setting. Connected mode only seems to like nice round > MTU values (eg. 32768 or 2048) so the default setting always gives > terrible performance. > > Datagram mode seems to have no problem with odd MTU values like 2044 > (This seems to be the max in Datagram mode). > > The other thing I noticed was Centos warns that multicast will be > broken with MTUs above 2044, this seems to be true, I don’t need it so > disabled multicast on the Infiniband interface. > > Is the behaviour I’m seeing normal or is this some kind of bug? > (Either in OFED running on Centos 5.2 or QPerf) Results with various > settings below. > > Rob > > Default (Connected mode) > > [root at mrhappy OFED-1.3.1]# qperf 192.168.10.161 tcp_bw tcp_lat > > tcp_bw: > > bw = 32.8 KB/sec > > tcp_lat: > > latency = 42.2 us > > Datagram Mode with 2044 MTU (Max for Datagram): > > [root at mrtickle OFED-1.3.1]# qperf 192.168.10.160 tcp_bw tcp_lat > > tcp_bw: > > bw = 400 MB/sec > > tcp_lat: > > latency = 41.2 us > > Connected Mode with 32768 MTU: > > [root at mrtickle ~]# qperf 192.168.10.160 tcp_bw tcp_lat > > tcp_bw: > > bw = 866 MB/sec > > tcp_lat: > > latency = 30.8 us > > ** > > ** > > *The SAQ Group* > > *Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ* > SEMTEC Limited Trading as SAQ is Registered in England & Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication and > I.T. services to UK Business. > > DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : > Backups : Managed Networks : Remote Support. > > Find us in http://www.thebestof.co.uk/petersfield > > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From dtur6utu126.com at lists.openfabrics.org Fri Jul 25 14:32:21 2008 From: dtur6utu126.com at lists.openfabrics.org (=?GB2312?B?wO7PyMn6?=) Date: Sat, 26 Jul 2008 05:32:21 +0800 Subject: [ofa-general] =?gb2312?b?L7eiL8axLw==?= Message-ID: <20080725213539.2E90AE60DD3@openfabrics.org> 你好 我公司有一部分发票可为贵公司代开 如： [商品销售 、广告 、服务 、运输 、建筑安装 、餐饮定额等发票代理 ]， 如需请联系： 负责人，李先生（0）13689556043 From flatif at neteffect.com Fri Jul 25 14:40:35 2008 From: flatif at neteffect.com (Faisal Latif) Date: Fri, 25 Jul 2008 16:40:35 -0500 Subject: [ofa-general] ***SPAM*** [PATCH v3] RDMA/nes: CM connection setup/teardown rework Message-ID: <200807252140.m6PLeZt8023645@velma.neteffect.com> Major rework of CM connection setup/teardown. We had a number of issues with MPI applications not starting/terminating properly over time. With these changes we were able to run longer on larger clusters. * Removed memory allocation from nes_connect() and nes_cm_connect(). * Fixed mini_cm_dec_refcnt_listen() when destroying listener. * Removed unnecessary code from schedule_nes_timer() and nes_cm_timer_tick(). * Functionalized mini_cm_recv_pkt() and process_packet(). * Cleaned up cm_node->ref_count usage. * Reuse skbs if available. Signed-off-by: Faisal Latif --- v3 - Delta patch to v2. Missed two changes to nes_cm.c that caused a BUG_ON in nes_destroy_qp(). diff -Nurp a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c --- a/drivers/infiniband/hw/nes/nes_cm.c 2008-07-24 05:23:57.000000000 -0500 +++ b/drivers/infiniband/hw/nes/nes_cm.c 2008-07-24 05:18:45.000000000 -0500 @@ -2837,6 +2837,7 @@ int nes_connect(struct iw_cm_id *cm_id, cm_node->apbvt_set = 1; nesqp->cm_node = cm_node; cm_node->nesqp = nesqp; + nes_add_ref(&nesqp->ibqp); return 0; } @@ -3167,7 +3168,6 @@ static void cm_event_connect_error(struc if (ret) printk(KERN_ERR "%s[%u] OFA CM event_handler returned, " "ret=%d\n", __func__, __LINE__, ret); - nes_rem_ref(&nesqp->ibqp); cm_id->rem_ref(cm_id); rem_ref_cm_node(event->cm_node->cm_core, event->cm_node); From mashirle at us.ibm.com Fri Jul 25 16:56:33 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Fri, 25 Jul 2008 16:56:33 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> On Fri, 2008-07-25 at 10:25 -0700, Roland Dreier wrote: > The current code at least tries to do this: it decrements the recv_count > on every receive completion with unsuccessful status, and moves an rx > countext to the reap list when its last receive completes. I don't see > any obvious bug here; is it not working in your tests? Right, it doesn't work. > > To use post_send last WR is much simple, just wait for this CQE to be > > generated in CQ then release the resource. The wait time here might be > > much smaller than above approach. > > But when do you post the send work request? This only works if you > already know the last receive has completed, because there is no > synchronization required between the receive queue and the send queue, > and so the completion of the send work request doesn't tell you anything > you didn't already know. > > - R. Do the post send work request whenever receiving DREQ, which meant the remote TX QP has already teared down, there is no new post recv completions any more after DREQ. Thanks Shirley From rdreier at cisco.com Fri Jul 25 17:36:12 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 25 Jul 2008 17:36:12 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Fri, 25 Jul 2008 16:56:33 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > > The current code at least tries to do this: it decrements the recv_count > > on every receive completion with unsuccessful status, and moves an rx > > countext to the reap list when its last receive completes. I don't see > > any obvious bug here; is it not working in your tests? > Right, it doesn't work. Do you have any further information about what goes wrong? > Do the post send work request whenever receiving DREQ, which meant the > remote TX QP has already teared down, there is no new post recv > completions any more after DREQ. After a DREQ is received, then the local QP is transitioned to the error state. However, we don't know when all the receives queued up have completed (with flush error status). Also, we may want to clean up a QP when we didn't receive a DREQ (remote side crashed, or we just have an idle connection). - R. From mashirle at us.ibm.com Fri Jul 25 18:07:57 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Fri, 25 Jul 2008 18:07:57 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217034477.12935.15.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, > > Do the post send work request whenever receiving DREQ, which meant the > > remote TX QP has already teared down, there is no new post recv > > completions any more after DREQ. > > After a DREQ is received, then the local QP is transitioned to the error > state. However, we don't know when all the receives queued up have > completed (with flush error status). Also, we may want to clean up a QP > when we didn't receive a DREQ (remote side crashed, or we just have an > idle connection). There are three senarios we need to destroy QPs. 1. During connection establishement 2. Received DREQ 3. Idle connection, or remote side crashed, shutdown Currently, IPoIB-CM uses the same approach to destroy QP for all these cases, which is putting the QP to error status, and wait for async event LAST WQE to destroy the QP from post send last WQ context. Putting QP to error status could return error, so async event might not be generated. What I want to propose here: 1. During connection establishment: like REJ, REP failure, we should destory the QP immediately without putting QP to error since the QP is not RTU. 2. Received DREQ, we should destroy it in post send WR last WR context, we don't need to reply on async event. 3. Idle conneciton or remote side crashed, shutdown, we should put the QP to error, then destroy later. And if number of connections run out, we can GC the less recent used connection for the new connection. Then we have a common approach for both nonSRQ and SRQ. We can remove async event handler. I have tested above patches. But I would like to hear your thoughts before sumitting them. Thanks Shirley From mashirle at us.ibm.com Fri Jul 25 19:29:12 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Fri, 25 Jul 2008 19:29:12 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217039352.12935.24.camel@IBM-29AB850785D.beaverton.ibm.com> On Fri, 2008-07-25 at 17:36 -0700, Roland Dreier wrote: > > > The current code at least tries to do this: it decrements the recv_count > > > on every receive completion with unsuccessful status, and moves an rx > > > countext to the reap list when its last receive completes. I don't see > > > any obvious bug here; is it not working in your tests? > > > Right, it doesn't work. > > Do you have any further information about what goes wrong? I don't see incoming receive completions with unsuccessful status in below status: 1. During connection establishement, 2. Received DREQ, remote TX QP has teared down, 3. Idle connection, or remote side crashed, shutdown, Which senarios do you think it appies? Thanks Shirley From vlad at lists.openfabrics.org Sat Jul 26 02:53:09 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sat, 26 Jul 2008 02:53:09 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080726-0200 daily build status Message-ID: <20080726095309.C7B9BE60943@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080726-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080726-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080726-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080726-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080726-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080726-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From sashak at voltaire.com Fri Jul 25 20:32:51 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 06:32:51 +0300 Subject: [ofa-general] ***SPAM*** [PATCH] opensm/configure.in: don't touch CFLAGS directly Message-ID: <20080726033251.GD14466@sashak.voltaire.com> Using this CFLAGS=$ac_env_CFLAGS_value is preventing -O2 from default CFLAGS. Signed-off-by: Sasha Khapyorsky --- opensm/configure.in | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/opensm/configure.in b/opensm/configure.in index a4f5b61..7d04c14 100644 --- a/opensm/configure.in +++ b/opensm/configure.in @@ -216,12 +216,6 @@ dnl Checks for headers and libraries OPENIB_APP_OSMV_CHECK_HEADER OPENIB_APP_OSMV_CHECK_LIB -# we have to revive the env CFLAGS as some how they are being overwritten... -# see http://sources.redhat.com/automake/automake.html#Flag-Variables-Ordering -# for why they should NEVER be modified by the configure to allow for user -# overrides. -CFLAGS=$ac_env_CFLAGS_value - AC_CONFIG_FILES([man/opensm.8 scripts/opensm.init scripts/redhat-opensm.init scripts/sldd.sh]) dnl Create the following Makefiles -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Sat Jul 26 07:47:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 17:47:39 +0300 Subject: [ofa-general] Re: [PATCH] ibsim/sim_mad.c: Cosmetic changes In-Reply-To: References: Message-ID: <20080726144739.GA5067@sashak.voltaire.com> On 09:05 Fri 25 Jul , Hal Rosenstock wrote: > ibsim/sim_mad.c: Cosmetic changes > > Fix typo in IBWARN message > Make attribute ID format consistent (in hex) > Also, other cosmetic formatting change > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Sat Jul 26 07:55:12 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 17:55:12 +0300 Subject: [ofa-general] Re: [PATCH] libibmad/ChangeLog: Fix typo In-Reply-To: References: Message-ID: <20080726145512.GB5067@sashak.voltaire.com> On 09:27 Fri 25 Jul , Hal Rosenstock wrote: > libibmad/ChangeLog: Fix typo > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Sat Jul 26 07:55:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 17:55:32 +0300 Subject: [ofa-general] Re: [PATCH] libibumad/umad.c: Cosmetic change In-Reply-To: References: Message-ID: <20080726145532.GC5067@sashak.voltaire.com> On 09:27 Fri 25 Jul , Hal Rosenstock wrote: > libibumad/umad.c: Cosmetic change > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From Celina at interkom.pl Sat Jul 26 08:33:00 2008 From: Celina at interkom.pl (Celina Reagan) Date: Sat, 26 Jul 2008 18:33:00 +0300 Subject: [ofa-general] Last news for Katheryn Sams Message-ID: <039b01c8ef34$e2a71e00$c0a80202@Celina> In New York has landed UFO. Click to see video! http://domrep.do.funpic.de/use_v/hot.php -------------- next part -------------- An HTML attachment was scrubbed... URL: From swise at opengridcomputing.com Sat Jul 26 08:58:08 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Sat, 26 Jul 2008 10:58:08 -0500 Subject: [ofa-general] Re: [ewg] OFED 1.4 alpha release is ready In-Reply-To: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> Message-ID: <488B4990.2010208@opengridcomputing.com> How can we claim an alpha release when the tree is still based on linux-2.6.26 and doesn't have a large chunk of the new features integrated? Tziporet Koren wrote: > Hi, > > OFED 1.4 alpha release is available on > http://www.openfabrics.org/downloads/OFED/ofed-1.4/OFED-1.4-alpha1.tgz > > To get BUILD_ID run ofed_info > > Please report any issues in bugzilla https://bugs.openfabrics.org/ for > OFED 1.4 > > The beta release is expected on August 7 2008 > > Tziporet & Vlad > > > ======================================================================== > > Release information: > -------------------- > Linux Operating Systems: > - RedHat EL4 up5: 2.6.9-55.ELsmp > - RedHat EL4 up6: 2.6.9-67.ELsmp > - RedHat EL5: 2.6.18-8.el5 > - RedHat EL5 up1: 2.6.18-53.el5 > - RedHat EL5 up2: 2.6.18-92.el5 > - Fedora C9: 2.6.25-14.fc9 * > - SLES10: 2.6.16.21-0.8-smp > - SLES10 SP1: 2.6.16.46-0.12-smp > - SLES10 SP1 up1: 2.6.16.53-0.16-smp > - SLES10 SP2: 2.6.16.60-0.21-smp > - OpenSuSE 10.3: 2.6.22.5-31 * > - kernel.org: 2.6.26 > > * OSes that are partially tested > > Systems: > * x86_64 > * x86 > * ia64 > * ppc64 > > > Main Changes from OFED 1.3 > ========================== > 1. General changes > o Kernel code based on 2.6.26 (will be 2.6.27) > o Added iSER target package > o Added NFS-RDMA support (for 2.6.26 only for now) > 2. IPoIB > o LRO support - when we move to 2.6.27 > 3. SDP > o Bug fixes in the state machine and close flow > 4. qlgc_vnic > o Support for hotswap of EVIC and dynamic update of existing > connections > with the addition of QLogic dynamic update daemon. > o Performance improvements in handling of Ethernet > broadcast/multicast > traffic. > 5. RDS (for now only kernel 2.6.26 is working) > o GA of RDMA API (using FMRs) - RDS API version 3 > o iWARP support > 6. uDAPL > o Added socket based CM - for both scalability and interop with > Windows > o Added UD extensions - for version 2.0 only > o v1 library package has been renamed to compat-dapl-1.2.8-1 > 7. Management > o OpenSM > - APM - disjoint paths > - Path balancing for LMC + console diagnostics > - OpenSM configuration unification > - MGID to MLID mapping for IPv6 SNM > - Routing engines chain > - IBA 1.2.1 additions > - Failover/Handover improvements > o ibutils: > - Congestion Control > - Report created in CSV format > o Diagnostic tools: > - ibnetdiscover library - to accelerate another tools > 8. Low level drivers: > o mlx4: Enable Virtual Protocol Interconnect - Eth and IB on the > same device > > > Tasks that should be completed for the beta: > ============================================ > 1. Move to kernel base 2.6.27-rc1 with the features: > - New verbs to support BMME (Fast memory thru send queue, Local > invalidate send work requests, > Read with invalidate. > - IPoIB LRO > 2. RDS to work on distro OSes > 3. NFS-RDMA to work on distro OSes > 4. New MPI versions: MVAPICH 1.1, OpenMPI 1.3, MVAPICH2 1.2 > 5. OSM: Cashed routing > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > From sashak at voltaire.com Sat Jul 26 10:40:25 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 20:40:25 +0300 Subject: [ofa-general] [PATCH] opensm/osm_ucast_lash.c: remove unused variables Message-ID: <20080726174025.GF5067@sashak.voltaire.com> From: Keshetti Mahesh This removes some unnecessary variables in some lash structures. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_lash.c | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c index 2e67c26..b3e410c 100644 --- a/opensm/opensm/osm_ucast_lash.c +++ b/opensm/opensm/osm_ucast_lash.c @@ -67,7 +67,6 @@ enum { }; typedef struct _cdg_vertex { - int switch_size; int num_dependencies; struct _cdg_vertex **dependency; int from; @@ -92,7 +91,6 @@ typedef struct _switch { int id; int used_channels; int q_state; - int dist; struct routing_table { unsigned out_link; unsigned lane; @@ -331,11 +329,10 @@ static void remove_semipermanent_depend_for_sp(lash_t * p_lash, int sw, } } -inline static void enqueue(cl_list_t * bfsq, switch_t * sw, int dist) +inline static void enqueue(cl_list_t * bfsq, switch_t * sw) { CL_ASSERT(sw->q_state == UNQUEUED); sw->q_state = Q_MEMBER; - sw->dist = dist; cl_list_insert_tail(bfsq, sw); } @@ -366,14 +363,14 @@ static void shortest_path(lash_t * p_lash, int ir) cl_list_construct(&bfsq); cl_list_init(&bfsq, 20); - enqueue(&bfsq, switches[ir], 0); + enqueue(&bfsq, switches[ir]); while (!cl_is_list_empty(&bfsq)) { dequeue(&bfsq, &sw); for (i = 0; i < sw->num_connections; i++) { swi = switches[sw->phys_connections[i]]; if (swi->q_state == UNQUEUED) { - enqueue(&bfsq, swi, sw->dist + 1); + enqueue(&bfsq, swi); sw->dij_channels[sw->used_channels++] = swi->id; } } -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Sat Jul 26 10:41:23 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 20:41:23 +0300 Subject: [ofa-general] [PATCH] opensm/osm_ucast_lash: remove an invalid error log In-Reply-To: <20080726174025.GF5067@sashak.voltaire.com> References: <20080726174025.GF5067@sashak.voltaire.com> Message-ID: <20080726174123.GG5067@sashak.voltaire.com> LID space may have a "holes", so if it is not an error if port is not resolved by random LID. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_lash.c | 10 ++-------- 1 files changed, 2 insertions(+), 8 deletions(-) diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c index b3e410c..bb929b6 100644 --- a/opensm/opensm/osm_ucast_lash.c +++ b/opensm/opensm/osm_ucast_lash.c @@ -1134,13 +1134,7 @@ static void populate_fwd_tbls(lash_t * p_lash) for (lid = 1; lid <= max_lid_ho; lid++) { p_dst_sw = get_osm_switch_from_lid(p_lash->p_osm, lid); - if (p_dst_sw == NULL) { - OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 4D03: " - "LASH fwd NULL Cannot find GUID 0x%016" - PRIx64 - " src lash id (%d), src lid no (0x%04X)\n", - cl_ntoh64(current_guid), sw->id, lid); - } else if (p_dst_sw == p_sw) { + if (p_dst_sw == p_sw) { uint8_t egress_port = find_port_from_lid(cl_hton16(lid), p_sw); p_osm->sm.ucast_mgr.lft_buf[lid] = egress_port; @@ -1152,7 +1146,7 @@ static void populate_fwd_tbls(lash_t * p_lash) cl_ntoh64(current_guid), -1, lid, egress_port, cl_ntoh64(current_guid), -1, egress_port); - } else { + } else if (p_dst_sw) { unsigned dst_lash_switch_id = get_lash_id(p_dst_sw); uint8_t lash_egress_port = -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Sat Jul 26 10:42:59 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 26 Jul 2008 20:42:59 +0300 Subject: [ofa-general] [PATCH] opensm/osm_ucast_lash: find_port_from_lid() function is not used In-Reply-To: <20080726174123.GG5067@sashak.voltaire.com> References: <20080726174025.GF5067@sashak.voltaire.com> <20080726174123.GG5067@sashak.voltaire.com> Message-ID: <20080726174259.GH5067@sashak.voltaire.com> This function is called only for the case when source and destination switches are the same, so it is routing to itself and port number is always '0'. The patch removes find_port_from_lid() function and its usage. Based on comments from: Keshetti Mahesh Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_lash.c | 62 ++-------------------------------------- 1 files changed, 3 insertions(+), 59 deletions(-) diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c index bb929b6..cc365db 100644 --- a/opensm/opensm/osm_ucast_lash.c +++ b/opensm/opensm/osm_ucast_lash.c @@ -153,59 +153,6 @@ static osm_switch_t *get_osm_switch_from_lid(osm_opensm_t * osm, uint16_t lid) return get_osm_switch_from_port(port); } -// This is a time consuming way to find a port from a lid -// we will come up with a better way later -static uint8_t find_port_from_lid(IN const ib_net16_t lid_no, - IN const osm_switch_t * p_sw) -{ - uint8_t port_count = 0; - uint8_t i = 0; - osm_physp_t *p_current_physp, *p_remote_physp = NULL; - ib_port_info_t *port_info; - ib_net16_t port_lid; - uint8_t egress_port = 255; - - if (p_sw->p_node) - port_count = osm_node_get_num_physp(p_sw->p_node); - - // process management port first - p_current_physp = osm_node_get_physp_ptr(p_sw->p_node, 0); - - port_info = &p_current_physp->port_info; - port_lid = port_info->base_lid; - if (port_lid == lid_no) { - egress_port = 0; - goto Exit; - } - // process each port on this switch - for (i = 1; i < port_count; i++) { - - p_current_physp = osm_node_get_physp_ptr(p_sw->p_node, i); - if (!p_current_physp) - continue; - - p_remote_physp = p_current_physp->p_remote_physp; - - if (p_remote_physp) { - osm_node_t *p_opposite_node = - osm_physp_get_node_ptr(p_remote_physp); - - if (osm_node_get_type(p_opposite_node) == - IB_NODE_TYPE_CA) { - ib_port_info_t *pi = &p_remote_physp->port_info; - ib_net16_t remote_port_lid = pi->base_lid; - if (remote_port_lid == lid_no) { - egress_port = i; - goto Exit; - } - } - } - } // for - -Exit: - return egress_port; -} - #if 0 static int randint(int high) { @@ -1135,17 +1082,14 @@ static void populate_fwd_tbls(lash_t * p_lash) p_dst_sw = get_osm_switch_from_lid(p_lash->p_osm, lid); if (p_dst_sw == p_sw) { - uint8_t egress_port = - find_port_from_lid(cl_hton16(lid), p_sw); - p_osm->sm.ucast_mgr.lft_buf[lid] = egress_port; + p_osm->sm.ucast_mgr.lft_buf[lid] = 0; OSM_LOG(p_log, OSM_LOG_VERBOSE, "LASH fwd MY SRC SRC GUID 0x%016" PRIx64 " src lash id (%d), src lid no (0x%04X) src lash port (%d) " "DST GUID 0x%016" PRIx64 " src lash id (%d), src lash port (%d)\n", - cl_ntoh64(current_guid), -1, lid, - egress_port, cl_ntoh64(current_guid), - -1, egress_port); + cl_ntoh64(current_guid), -1, lid, 0, + cl_ntoh64(current_guid), -1, 0); } else if (p_dst_sw) { unsigned dst_lash_switch_id = get_lash_id(p_dst_sw); -- 1.5.5.1.178.g1f811 From rdreier at cisco.com Sat Jul 26 14:02:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 26 Jul 2008 14:02:11 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get the some last RDMA/InfiniBand changes for 2.6.27. The only major thing is changes to the iw_nes connection setup handling, which fix problems with lots of connections: Faisal Latif (1): RDMA/nes: CM connection setup/teardown rework Jack Morgenstein (1): mlx4: Update/add Mellanox Technologies copyright lines to mlx4 driver files Roland Dreier (5): RDMA/ucma: BKL is not needed for ucma_open() RDMA/ucm: BKL is not needed for ib_ucm_open() IPoIB/cm: Connected mode is no longer EXPERIMENTAL IPoIB: Correct help text for INFINIBAND_IPOIB_DEBUG Merge branches 'bkl-removal', 'ipoib', 'mlx4' and 'nes' into for-linus Yevgeny Petrilin (1): mlx4_core: Add VLAN tag field to WQE control segment struct drivers/infiniband/core/ucm.c | 10 +- drivers/infiniband/core/ucma.c | 11 +- drivers/infiniband/hw/mlx4/cq.c | 1 + drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 1 + drivers/infiniband/hw/mlx4/mr.c | 1 + drivers/infiniband/hw/mlx4/qp.c | 1 + drivers/infiniband/hw/mlx4/srq.c | 1 + drivers/infiniband/hw/mlx4/user.h | 1 + drivers/infiniband/hw/nes/nes.c | 4 +- drivers/infiniband/hw/nes/nes_cm.c | 2034 ++++++++++++++++++--------------- drivers/infiniband/hw/nes/nes_cm.h | 23 +- drivers/infiniband/hw/nes/nes_hw.c | 9 - drivers/infiniband/hw/nes/nes_verbs.c | 15 - drivers/infiniband/ulp/ipoib/Kconfig | 22 +- drivers/net/mlx4/alloc.c | 1 + drivers/net/mlx4/catas.c | 1 + drivers/net/mlx4/cmd.c | 2 +- drivers/net/mlx4/cq.c | 2 +- drivers/net/mlx4/eq.c | 2 +- drivers/net/mlx4/fw.c | 2 +- drivers/net/mlx4/fw.h | 2 +- drivers/net/mlx4/icm.c | 2 +- drivers/net/mlx4/icm.h | 2 +- drivers/net/mlx4/intf.c | 1 + drivers/net/mlx4/main.c | 2 +- drivers/net/mlx4/mcg.c | 1 + drivers/net/mlx4/mlx4.h | 2 +- drivers/net/mlx4/mr.c | 2 +- drivers/net/mlx4/qp.c | 2 +- drivers/net/mlx4/reset.c | 1 + drivers/net/mlx4/srq.c | 1 + include/linux/mlx4/qp.h | 4 +- 33 files changed, 1211 insertions(+), 956 deletions(-) From rdreier at cisco.com Sat Jul 26 14:25:32 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 26 Jul 2008 14:25:32 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1217034477.12935.15.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Fri, 25 Jul 2008 18:07:57 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217034477.12935.15.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > Then we have a common approach for both nonSRQ and SRQ. We can remove > async event handler. I have tested above patches. But I would like to > hear your thoughts before sumitting them. Seems like it would end up ignoring ordering rules for handling completions etc, and open windows for either leaking receive buffers or use-after-free. I don't know what the motivation is: what problems do we solve by doing this? From rdreier at cisco.com Sat Jul 26 14:26:38 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 26 Jul 2008 14:26:38 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1217039352.12935.24.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Fri, 25 Jul 2008 19:29:12 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217039352.12935.24.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > > Do you have any further information about what goes wrong? > I don't see incoming receive completions with unsuccessful status in > below status: > 1. During connection establishement, > 2. Received DREQ, remote TX QP has teared down, > 3. Idle connection, or remote side crashed, shutdown, I don't understand... are you saying that pending receives don't get completed with a flush error when a QP transitions to error state? - R. From sashak at voltaire.com Sat Jul 26 19:03:06 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 05:03:06 +0300 Subject: [ofa-general] [PATCH] opensm: remove osm_log_is_active() check In-Reply-To: <20080709223534.GA12551@sashak.voltaire.com> References: <20080709223030.GY12551@sashak.voltaire.com> <20080709223132.GZ12551@sashak.voltaire.com> <20080709223534.GA12551@sashak.voltaire.com> Message-ID: <20080727020306.GJ5067@sashak.voltaire.com> This osm_log_is_active() check is built in OSM_LOG() macro now and can be cleaned up from the code. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_drop_mgr.c | 43 ++++------- opensm/opensm/osm_dump.c | 3 +- opensm/opensm/osm_inform.c | 7 +-- opensm/opensm/osm_lid_mgr.c | 46 ++++------- opensm/opensm/osm_link_mgr.c | 18 ++--- opensm/opensm/osm_mcast_fwd_rcv.c | 12 +-- opensm/opensm/osm_mcast_mgr.c | 132 +++++++++++-------------------- opensm/opensm/osm_node_desc_rcv.c | 9 +-- opensm/opensm/osm_node_info_rcv.c | 27 +++---- opensm/opensm/osm_pkey_mgr.c | 3 +- opensm/opensm/osm_pkey_rcv.c | 12 +-- opensm/opensm/osm_port.c | 74 +++++++---------- opensm/opensm/osm_port_info_rcv.c | 55 +++++-------- opensm/opensm/osm_req.c | 22 ++--- opensm/opensm/osm_resp.c | 11 +-- opensm/opensm/osm_sa_guidinfo_record.c | 26 ++---- opensm/opensm/osm_sa_informinfo.c | 6 +- opensm/opensm/osm_sa_lft_record.c | 11 +-- opensm/opensm/osm_sa_link_record.c | 14 ++-- opensm/opensm/osm_sa_mad_ctrl.c | 15 +--- opensm/opensm/osm_sa_mft_record.c | 11 +-- opensm/opensm/osm_sa_multipath_record.c | 71 +++++++---------- opensm/opensm/osm_sa_node_record.c | 51 ++++-------- opensm/opensm/osm_sa_path_record.c | 59 ++++++--------- opensm/opensm/osm_sa_pkey_record.c | 12 +-- opensm/opensm/osm_sa_portinfo_record.c | 18 ++--- opensm/opensm/osm_sa_service_record.c | 21 ++--- opensm/opensm/osm_sa_slvl_record.c | 12 +-- opensm/opensm/osm_sa_sminfo_record.c | 5 +- opensm/opensm/osm_sa_sw_info_record.c | 19 ++--- opensm/opensm/osm_sa_vlarb_record.c | 11 +-- opensm/opensm/osm_slvl_map_rcv.c | 13 +-- opensm/opensm/osm_sm_mad_ctrl.c | 58 +++++--------- opensm/opensm/osm_sm_state_mgr.c | 18 ++--- opensm/opensm/osm_sminfo_rcv.c | 7 +- opensm/opensm/osm_state_mgr.c | 35 +------- opensm/opensm/osm_sw_info_rcv.c | 33 +++----- opensm/opensm/osm_ucast_mgr.c | 54 +++++-------- opensm/opensm/osm_vl15intf.c | 50 +++++------- opensm/opensm/osm_vl_arb_rcv.c | 12 +-- 40 files changed, 423 insertions(+), 693 deletions(-) diff --git a/opensm/opensm/osm_drop_mgr.c b/opensm/opensm/osm_drop_mgr.c index a7b2553..1aeb172 100644 --- a/opensm/opensm/osm_drop_mgr.c +++ b/opensm/opensm/osm_drop_mgr.c @@ -251,14 +251,12 @@ static void __osm_drop_mgr_remove_port(osm_sm_t * sm, IN osm_port_t * p_port) goto Exit; } - if (osm_log_is_active(sm->p_log, OSM_LOG_INFO)) { - OSM_LOG(sm->p_log, OSM_LOG_INFO, - "Removed port with GUID:0x%016" PRIx64 - " LID range [%u, %u] of node:%s\n", - cl_ntoh64(port_gid.unicast.interface_id), - min_lid_ho, max_lid_ho, - p_node ? p_node->print_desc : "UNKNOWN"); - } + OSM_LOG(sm->p_log, OSM_LOG_INFO, + "Removed port with GUID:0x%016" PRIx64 + " LID range [%u, %u] of node:%s\n", + cl_ntoh64(port_gid.unicast.interface_id), + min_lid_ho, max_lid_ho, + p_node ? p_node->print_desc : "UNKNOWN"); Exit: OSM_LOG_EXIT(sm->p_log); @@ -428,8 +426,6 @@ void osm_drop_mgr_process(osm_sm_t * sm) osm_port_t *p_next_port; osm_node_t *p_node; osm_node_t *p_next_node; - ib_net64_t port_guid; - ib_net64_t node_guid; CL_ASSERT(sm); @@ -449,12 +445,9 @@ void osm_drop_mgr_process(osm_sm_t * sm) CL_ASSERT(cl_qmap_key(&p_node->map_item) == osm_node_get_node_guid(p_node)); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - node_guid = osm_node_get_node_guid(p_node); - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Checking node 0x%016" PRIx64 "\n", - cl_ntoh64(node_guid)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Checking node 0x%016" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); /* Check if this node was discovered during the last sweep. @@ -478,12 +471,9 @@ void osm_drop_mgr_process(osm_sm_t * sm) p_next_node = (osm_node_t *) cl_qmap_next(&p_next_node->map_item); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - node_guid = osm_node_get_node_guid(p_node); - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Checking full discovery of node 0x%016" PRIx64 - "\n", cl_ntoh64(node_guid)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Checking full discovery of node 0x%016" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); if (osm_node_get_type(p_node) != IB_NODE_TYPE_SWITCH) continue; @@ -501,12 +491,9 @@ void osm_drop_mgr_process(osm_sm_t * sm) CL_ASSERT(cl_qmap_key(&p_port->map_item) == osm_port_get_guid(p_port)); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - port_guid = osm_port_get_guid(p_port); - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Checking port 0x%016" PRIx64 "\n", - cl_ntoh64(port_guid)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Checking port 0x%016" PRIx64 "\n", + cl_ntoh64(osm_port_get_guid(p_port))); /* If the port is unreachable, remove it from the guid table. diff --git a/opensm/opensm/osm_dump.c b/opensm/opensm/osm_dump.c index 9447cc8..7e0ba47 100644 --- a/opensm/opensm/osm_dump.c +++ b/opensm/opensm/osm_dump.c @@ -606,12 +606,11 @@ static void print_report(osm_opensm_t *osm, FILE *file) void osm_dump_mcast_routes(osm_opensm_t * osm) { - if (osm_log_is_active(&osm->log, OSM_LOG_ROUTING)) { + if (osm_log_is_active(&osm->log, OSM_LOG_ROUTING)) /* multicast routes */ osm_dump_qmap_to_file(osm, "opensm.mcfdbs", &osm->subn.sw_guid_tbl, dump_mcast_routes, osm); - } } void osm_dump_all(osm_opensm_t * osm) diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index c8c06b2..b9a3254 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -85,10 +85,8 @@ static void dump_all_informs(IN osm_subn_t const *p_subn, IN osm_log_t * p_log) { cl_list_item_t *p_list_item; - OSM_LOG_ENTER(p_log); - if (!osm_log_is_active(p_log, OSM_LOG_DEBUG)) - goto Exit; + return; p_list_item = cl_qlist_head(&p_subn->sa_infr_list); while (p_list_item != cl_qlist_end(&p_subn->sa_infr_list)) { @@ -97,9 +95,6 @@ static void dump_all_informs(IN osm_subn_t const *p_subn, IN osm_log_t * p_log) inform_record.inform_info, OSM_LOG_DEBUG); p_list_item = cl_qlist_next(p_list_item); } - -Exit: - OSM_LOG_EXIT(p_log); } /********************************************************************** diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c index d66ced3..9472065 100644 --- a/opensm/opensm/osm_lid_mgr.c +++ b/opensm/opensm/osm_lid_mgr.c @@ -656,8 +656,7 @@ __osm_lid_mgr_find_free_lid_range(IN osm_lid_mgr_t * const p_mgr, uint8_t lmc_num_lids; uint16_t lmc_mask; - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "LMC = %u, number LIDs = %u\n", + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, "LMC = %u, number LIDs = %u\n", p_mgr->p_subn->opt.lmc, num_lids); lmc_num_lids = (1 << p_mgr->p_subn->opt.lmc); @@ -772,10 +771,8 @@ __osm_lid_mgr_get_port_lid(IN osm_lid_mgr_t * const p_mgr, *p_min_lid = min_lid; *p_max_lid = min_lid + num_lids - 1; if (min_lid == cl_ntoh16(osm_port_get_base_lid(p_port))) { - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "0x%016" PRIx64 - " matches its known lid:%u\n", guid, - min_lid); + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, "0x%016" PRIx64 + " matches its known lid:%u\n", guid, min_lid); goto Exit; } else { OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, @@ -906,11 +903,9 @@ __osm_lid_mgr_set_physp_pi(IN osm_lid_mgr_t * const p_mgr, with the following attributes as they are set later (during NO_CHANGE state in link mgr). */ - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Skipping switch port %u, GUID 0x%016" PRIx64 - "\n", port_num, - cl_ntoh64(osm_physp_get_port_guid(p_physp))); + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Skipping switch port %u, GUID 0x%016" PRIx64 "\n", + port_num, cl_ntoh64(osm_physp_get_port_guid(p_physp))); goto Exit; } @@ -1038,17 +1033,14 @@ __osm_lid_mgr_set_physp_pi(IN osm_lid_mgr_t * const p_mgr, */ if ((mtu != ib_port_info_get_neighbor_mtu(p_old_pi)) || (op_vls != ib_port_info_get_op_vls(p_old_pi))) { - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Sending Link Down to GUID 0x%016" - PRIx64 "port %d due to op_vls or " - "mtu change. MTU:%u,%u VL_CAP:%u,%u\n", - cl_ntoh64(osm_physp_get_port_guid - (p_physp)), port_num, mtu, - ib_port_info_get_neighbor_mtu(p_old_pi), - op_vls, - ib_port_info_get_op_vls(p_old_pi) - ); + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Sending Link Down to GUID 0x%016" + PRIx64 "port %d due to op_vls or " + "mtu change. MTU:%u,%u VL_CAP:%u,%u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + port_num, mtu, + ib_port_info_get_neighbor_mtu(p_old_pi), + op_vls, ib_port_info_get_op_vls(p_old_pi)); /* we need to make sure the internal DB will follow the @@ -1186,12 +1178,10 @@ __osm_lid_mgr_process_our_sm_node(IN osm_lid_mgr_t * const p_mgr) p_mgr->p_subn->master_sm_base_lid = cl_hton16(min_lid_ho); p_mgr->p_subn->sm_base_lid = cl_hton16(min_lid_ho); - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_VERBOSE)) - OSM_LOG(p_mgr->p_log, OSM_LOG_VERBOSE, - "Assigning SM's port 0x%016" PRIx64 - "\n\t\t\t\tto LID range [%u,%u]\n", - cl_ntoh64(osm_port_get_guid(p_port)), - min_lid_ho, max_lid_ho); + OSM_LOG(p_mgr->p_log, OSM_LOG_VERBOSE, + "Assigning SM's port 0x%016" PRIx64 + "\n\t\t\t\tto LID range [%u,%u]\n", + cl_ntoh64(osm_port_get_guid(p_port)), min_lid_ho, max_lid_ho); /* Set the PortInfo the Physical Port associated with this Port. diff --git a/opensm/opensm/osm_link_mgr.c b/opensm/opensm/osm_link_mgr.c index 3454024..452d151 100644 --- a/opensm/opensm/osm_link_mgr.c +++ b/opensm/opensm/osm_link_mgr.c @@ -95,12 +95,9 @@ __osm_link_mgr_set_physp_pi(osm_sm_t * sm, == FALSE) { /* This means the switch doesn't support enhanced port 0. Can skip it. */ - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Skipping port 0, GUID 0x%016" PRIx64 - "\n", - cl_ntoh64(osm_physp_get_port_guid - (p_physp))); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Skipping port 0, GUID 0x%016" PRIx64 "\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp))); goto Exit; } esp0 = TRUE; @@ -347,11 +344,10 @@ __osm_link_mgr_process_node(osm_sm_t * sm, OSM_LOG_ENTER(sm->p_log); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Node 0x%" PRIx64 " going to %s\n", - cl_ntoh64(osm_node_get_node_guid(p_node)), - ib_get_port_state_str(link_state)); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Node 0x%" PRIx64 " going to %s\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + ib_get_port_state_str(link_state)); /* Set the PortInfo for every Physical Port associated diff --git a/opensm/opensm/osm_mcast_fwd_rcv.c b/opensm/opensm/osm_mcast_fwd_rcv.c index b2f491f..5271f69 100644 --- a/opensm/opensm/osm_mcast_fwd_rcv.c +++ b/opensm/opensm/osm_mcast_fwd_rcv.c @@ -89,13 +89,11 @@ void osm_mft_rcv_process(IN void *context, IN void *data) p_mft_context = osm_madw_get_mft_context_ptr(p_madw); node_guid = p_mft_context->node_guid; - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Setting MFT block %u, position %u, " - "Switch 0x%016" PRIx64 ", TID 0x%" PRIx64 "\n", - block_num, position, cl_ntoh64(node_guid), - cl_ntoh64(p_smp->trans_id)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Setting MFT block %u, position %u, " + "Switch 0x%016" PRIx64 ", TID 0x%" PRIx64 "\n", + block_num, position, cl_ntoh64(node_guid), + cl_ntoh64(p_smp->trans_id)); CL_PLOCK_EXCL_ACQUIRE(sm->p_lock); p_sw = osm_get_switch_by_guid(sm->p_subn, node_guid); diff --git a/opensm/opensm/osm_mcast_mgr.c b/opensm/opensm/osm_mcast_mgr.c index 20fb41f..78eed50 100644 --- a/opensm/opensm/osm_mcast_mgr.c +++ b/opensm/opensm/osm_mcast_mgr.c @@ -260,7 +260,6 @@ static osm_switch_t *__osm_mcast_mgr_find_optimal_switch(osm_sm_t * sm, const osm_switch_t *p_best_sw = NULL; float hops = 0; float best_hops = 10000; /* any big # will do */ - uint64_t sw_guid_ho; #ifdef OSM_VENDOR_INTF_ANAFA boolean_t use_avg_hops = TRUE; /* anafa2 - bug hca on switch *//* use max hops for root */ #else @@ -284,13 +283,9 @@ static osm_switch_t *__osm_mcast_mgr_find_optimal_switch(osm_sm_t * sm, else hops = osm_mcast_mgr_compute_max_hops(sm, p_mgrp, p_sw); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - sw_guid_ho = - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)); - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Switch 0x%016" PRIx64 ", hops = %f\n", - sw_guid_ho, hops); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Switch 0x%016" PRIx64 ", hops = %f\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), hops); if (hops < best_hops) { p_best_sw = p_sw; @@ -298,19 +293,14 @@ static osm_switch_t *__osm_mcast_mgr_find_optimal_switch(osm_sm_t * sm, } } - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - if (p_best_sw) { - sw_guid_ho = - cl_ntoh64(osm_node_get_node_guid - (p_best_sw->p_node)); - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Best switch is 0x%" PRIx64 ", hops = %f\n", - sw_guid_ho, best_hops); - } else { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "No multicast capable switches detected\n"); - } - } + if (p_best_sw) + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Best switch is 0x%" PRIx64 ", hops = %f\n", + cl_ntoh64(osm_node_get_node_guid(p_best_sw->p_node)), + best_hops); + else + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "No multicast capable switches detected\n"); OSM_LOG_EXIT(sm->p_log); return ((osm_switch_t *) p_best_sw); @@ -384,10 +374,8 @@ __osm_mcast_mgr_set_tbl(osm_sm_t * sm, IN osm_switch_t * const p_sw) while (osm_mcast_tbl_get_block(p_tbl, block_num, (uint8_t) position, block)) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Writing MFT block 0x%X\n", block_id_ho); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Writing MFT block 0x%X\n", block_id_ho); block_id_ho = block_num + (position << 28); @@ -554,12 +542,10 @@ static osm_mtree_node_t *__osm_mcast_mgr_branch(osm_sm_t * sm, node_guid_ho = cl_ntoh64(node_guid); mlid_ho = cl_ntoh16(osm_mgrp_get_mlid(p_mgrp)); - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Routing MLID 0x%X through switch 0x%" PRIx64 - ", %u nodes at depth %u\n", - mlid_ho, node_guid_ho, cl_qlist_count(p_list), depth); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Routing MLID 0x%X through switch 0x%" PRIx64 + ", %u nodes at depth %u\n", + mlid_ho, node_guid_ho, cl_qlist_count(p_list), depth); CL_ASSERT(cl_qlist_count(p_list) > 0); @@ -641,10 +627,8 @@ static osm_mtree_node_t *__osm_mcast_mgr_branch(osm_sm_t * sm, we're at the root of the spanning tree. */ if (depth > 1) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Adding upstream port 0x%X\n", upstream_port); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Adding upstream port 0x%X\n", upstream_port); CL_ASSERT(upstream_port); osm_mcast_tbl_set(p_tbl, mlid_ho, upstream_port); @@ -677,11 +661,9 @@ static osm_mtree_node_t *__osm_mcast_mgr_branch(osm_sm_t * sm, if (count == 0) continue; /* No routes down this port. */ - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Routing %zu destinations via switch port 0x%X\n", - count, i); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Routing %zu destinations via switch port 0x%X\n", + count, i); /* This port routes frames for this mcast group. Therefore, @@ -732,13 +714,11 @@ static osm_mtree_node_t *__osm_mcast_mgr_branch(osm_sm_t * sm, CL_ASSERT(cl_is_qlist_empty(p_port_list)); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Found leaf for port 0x%016" PRIx64 - " on switch port 0x%X\n", - cl_ntoh64(osm_port_get_guid - (p_wobj->p_port)), i); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Found leaf for port 0x%016" PRIx64 + " on switch port 0x%X\n", + cl_ntoh64(osm_port_get_guid (p_wobj->p_port)), + i); __osm_mcast_work_obj_delete(p_wobj); } @@ -781,11 +761,9 @@ __osm_mcast_mgr_build_spanning_tree(osm_sm_t * sm, osm_mgrp_t * const p_mgrp) p_mcm_tbl = &p_mgrp->mcm_port_tbl; num_ports = cl_qmap_count(p_mcm_tbl); if (num_ports == 0) { - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "MLID 0x%X has no members - nothing to do\n", - cl_ntoh16(osm_mgrp_get_mlid(p_mgrp))); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "MLID 0x%X has no members - nothing to do\n", + cl_ntoh16(osm_mgrp_get_mlid(p_mgrp))); goto Exit; } @@ -884,11 +862,9 @@ osm_mcast_mgr_set_table(osm_sm_t * sm, CL_ASSERT(p_sw); - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Configuring MLID 0x%X on switch 0x%" PRIx64 "\n", - mlid_ho, osm_node_get_node_guid(p_sw->p_node)); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Configuring MLID 0x%X on switch 0x%" PRIx64 "\n", + mlid_ho, osm_node_get_node_guid(p_sw->p_node)); /* For every child of this tree node, set the corresponding @@ -966,12 +942,10 @@ osm_mcast_mgr_process_single(osm_sm_t * sm, mlid_ho = cl_ntoh16(mlid); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Attempting to add port 0x%" PRIx64 " to MLID 0x%X, " - "\n\t\t\t\tjoin state = 0x%X\n", - cl_ntoh64(port_guid), mlid_ho, join_state); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Attempting to add port 0x%" PRIx64 " to MLID 0x%X, " + "\n\t\t\t\tjoin state = 0x%X\n", + cl_ntoh64(port_guid), mlid_ho, join_state); /* Acquire the Port object. @@ -1043,13 +1017,11 @@ osm_mcast_mgr_process_single(osm_sm_t * sm, osm_switch_get_mcast_tbl_ptr(p_remote_node->sw); osm_mcast_tbl_set(p_mcast_tbl, mlid_ho, port_num); } else { - if (join_state & IB_JOIN_STATE_SEND_ONLY) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Success. Nothing to do for send" - "only member\n"); - } - } else { + if (join_state & IB_JOIN_STATE_SEND_ONLY) + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Success. Nothing to do for send" + "only member\n"); + else { OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0A13: " "Unknown join state 0x%X\n", join_state); @@ -1057,12 +1029,8 @@ osm_mcast_mgr_process_single(osm_sm_t * sm, goto Exit; } } - } else { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Unable to add port\n"); - } - } + } else + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, "Unable to add port\n"); Exit: OSM_LOG_EXIT(sm->p_log); @@ -1086,19 +1054,15 @@ osm_mcast_mgr_process_tree(osm_sm_t * sm, mlid = osm_mgrp_get_mlid(p_mgrp); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Processing multicast group 0x%X\n", cl_ntoh16(mlid)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Processing multicast group 0x%X\n", cl_ntoh16(mlid)); /* If there are no switches in the subnet, then we have nothing to do. */ if (cl_qmap_count(&sm->p_subn->sw_guid_tbl) == 0) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "No switches in subnet. Nothing to do\n"); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "No switches in subnet. Nothing to do\n"); goto Exit; } diff --git a/opensm/opensm/osm_node_desc_rcv.c b/opensm/opensm/osm_node_desc_rcv.c index d848513..c0c64d7 100644 --- a/opensm/opensm/osm_node_desc_rcv.c +++ b/opensm/opensm/osm_node_desc_rcv.c @@ -81,12 +81,9 @@ __osm_nd_rcv_process_nd(IN osm_sm_t * sm, free(p_node->print_desc); p_node->print_desc = tmp_desc; - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Node 0x%" PRIx64 "\n\t\t\t\tDescription = %s\n", - cl_ntoh64(osm_node_get_node_guid(p_node)), - p_node->print_desc); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Node 0x%" PRIx64 "\n\t\t\t\tDescription = %s\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), p_node->print_desc); OSM_LOG_EXIT(sm->p_log); } diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 2bccf33..86710d1 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -233,14 +233,12 @@ __osm_ni_rcv_set_links(IN osm_sm_t * sm, } } - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Creating new link between:\n\t\t\t\tnode 0x%" PRIx64 - ", port number 0x%X and\n\t\t\t\tnode 0x%" PRIx64 - ", port number 0x%X\n", - cl_ntoh64(osm_node_get_node_guid(p_node)), port_num, - cl_ntoh64(p_ni_context->node_guid), - p_ni_context->port_num); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Creating new link between:\n\t\t\t\tnode 0x%" PRIx64 + ", port number 0x%X and\n\t\t\t\tnode 0x%" PRIx64 + ", port number 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), port_num, + cl_ntoh64(p_ni_context->node_guid), p_ni_context->port_num); osm_node_link(p_node, port_num, p_neighbor_node, p_ni_context->port_num); @@ -751,13 +749,12 @@ __osm_ni_rcv_process_existing(IN osm_sm_t * sm, p_ni_context = osm_madw_get_ni_context_ptr(p_madw); port_num = ib_node_info_get_local_port_num(p_ni); - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Rediscovered %s node 0x%" PRIx64 - " TID 0x%" PRIx64 ", discovered %u times already\n", - ib_get_node_type_str(p_ni->node_type), - cl_ntoh64(p_ni->node_guid), - cl_ntoh64(p_smp->trans_id), p_node->discovery_count); + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Rediscovered %s node 0x%" PRIx64 " TID 0x%" PRIx64 + ", discovered %u times already\n", + ib_get_node_type_str(p_ni->node_type), + cl_ntoh64(p_ni->node_guid), + cl_ntoh64(p_smp->trans_id), p_node->discovery_count); /* If we haven't already encountered this existing node diff --git a/opensm/opensm/osm_pkey_mgr.c b/opensm/opensm/osm_pkey_mgr.c index 04fb18b..a8f5100 100644 --- a/opensm/opensm/osm_pkey_mgr.c +++ b/opensm/opensm/osm_pkey_mgr.c @@ -461,13 +461,12 @@ pkey_mgr_update_peer_port(osm_log_t * p_log, osm_sm_t * sm, } } - if ((ret_val == TRUE) && osm_log_is_active(p_log, OSM_LOG_DEBUG)) { + if (ret_val) OSM_LOG(p_log, OSM_LOG_DEBUG, "Pkey table was updated for node 0x%016" PRIx64 " port %u\n", cl_ntoh64(osm_node_get_node_guid(p_node)), osm_physp_get_port_num(peer)); - } if (port_info_set) return TRUE; diff --git a/opensm/opensm/osm_pkey_rcv.c b/opensm/opensm/osm_pkey_rcv.c index db9ab99..9e09806 100644 --- a/opensm/opensm/osm_pkey_rcv.c +++ b/opensm/opensm/osm_pkey_rcv.c @@ -115,13 +115,11 @@ void osm_pkey_rcv_process(IN void *context, IN void *data) We do not mind if this is a result of a set or get - all we want is to update the subnet. */ - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Got GetResp(PKey) block:%u port_num %u with GUID 0x%" - PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" - PRIx64 "\n", block_num, port_num, cl_ntoh64(port_guid), - cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Got GetResp(PKey) block:%u port_num %u with GUID 0x%" + PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" + PRIx64 "\n", block_num, port_num, cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); /* Determine if we encountered a new Physical Port. diff --git a/opensm/opensm/osm_port.c b/opensm/opensm/osm_port.c index 4b71fa9..54a5fd1 100644 --- a/opensm/opensm/osm_port.c +++ b/opensm/opensm/osm_port.c @@ -321,33 +321,26 @@ osm_physp_calc_link_mtu(IN osm_log_t * p_log, IN const osm_physp_t * p_physp) remote_mtu = ib_port_info_get_mtu_cap(&p_remote_physp->port_info); - if (osm_log_is_active(p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_log, OSM_LOG_DEBUG, - "Remote port 0x%016" PRIx64 " port = 0x%X : " - "MTU = %u. This Port MTU: %u\n", - cl_ntoh64(osm_physp_get_port_guid - (p_remote_physp)), - osm_physp_get_port_num(p_remote_physp), - remote_mtu, mtu); + OSM_LOG(p_log, OSM_LOG_DEBUG, + "Remote port 0x%016" PRIx64 " port = 0x%X : " + "MTU = %u. This Port MTU: %u\n", + cl_ntoh64(osm_physp_get_port_guid(p_remote_physp)), + osm_physp_get_port_num(p_remote_physp), + remote_mtu, mtu); if (mtu != remote_mtu) { if (mtu > remote_mtu) mtu = remote_mtu; - if (osm_log_is_active(p_log, OSM_LOG_VERBOSE)) - OSM_LOG(p_log, OSM_LOG_VERBOSE, - "MTU mismatch between ports." - "\n\t\t\t\tPort 0x%016" PRIx64 - ", port 0x%X" " and port 0x%016" PRIx64 - ", port 0x%X." - "\n\t\t\t\tUsing lower MTU of %u\n", - cl_ntoh64(osm_physp_get_port_guid - (p_physp)), - osm_physp_get_port_num(p_physp), - cl_ntoh64(osm_physp_get_port_guid - (p_remote_physp)), - osm_physp_get_port_num(p_remote_physp), - mtu); + OSM_LOG(p_log, OSM_LOG_VERBOSE, + "MTU mismatch between ports." + "\n\t\t\t\tPort 0x%016" PRIx64 ", port 0x%X" + " and port 0x%016" PRIx64 ", port 0x%X." + "\n\t\t\t\tUsing lower MTU of %u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + osm_physp_get_port_num(p_physp), + cl_ntoh64(osm_physp_get_port_guid(p_remote_physp)), + osm_physp_get_port_num(p_remote_physp),mtu); } } else mtu = ib_port_info_get_neighbor_mtu(&p_physp->port_info); @@ -383,33 +376,26 @@ osm_physp_calc_link_op_vls(IN osm_log_t * p_log, remote_op_vls = ib_port_info_get_vl_cap(&p_remote_physp->port_info); - if (osm_log_is_active(p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_log, OSM_LOG_DEBUG, - "Remote port 0x%016" PRIx64 " port = 0x%X : " - "VL_CAP = %u. This port VL_CAP = %u\n", - cl_ntoh64(osm_physp_get_port_guid - (p_remote_physp)), - osm_physp_get_port_num(p_remote_physp), - remote_op_vls, op_vls); + OSM_LOG(p_log, OSM_LOG_DEBUG, + "Remote port 0x%016" PRIx64 " port = 0x%X : " + "VL_CAP = %u. This port VL_CAP = %u\n", + cl_ntoh64(osm_physp_get_port_guid(p_remote_physp)), + osm_physp_get_port_num(p_remote_physp), + remote_op_vls, op_vls); if (op_vls != remote_op_vls) { if (op_vls > remote_op_vls) op_vls = remote_op_vls; - if (osm_log_is_active(p_log, OSM_LOG_VERBOSE)) - OSM_LOG(p_log, OSM_LOG_VERBOSE, - "OP_VLS mismatch between ports." - "\n\t\t\t\tPort 0x%016" PRIx64 - ", port 0x%X" " and port 0x%016" PRIx64 - ", port 0x%X." - "\n\t\t\t\tUsing lower OP_VLS of %u\n", - cl_ntoh64(osm_physp_get_port_guid - (p_physp)), - osm_physp_get_port_num(p_physp), - cl_ntoh64(osm_physp_get_port_guid - (p_remote_physp)), - osm_physp_get_port_num(p_remote_physp), - op_vls); + OSM_LOG(p_log, OSM_LOG_VERBOSE, + "OP_VLS mismatch between ports." + "\n\t\t\t\tPort 0x%016" PRIx64 ", port 0x%X" + " and port 0x%016" PRIx64 ", port 0x%X." + "\n\t\t\t\tUsing lower OP_VLS of %u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + osm_physp_get_port_num(p_physp), + cl_ntoh64(osm_physp_get_port_guid(p_remote_physp)), + osm_physp_get_port_num(p_remote_physp), op_vls); } } else op_vls = ib_port_info_get_op_vls(&p_physp->port_info); diff --git a/opensm/opensm/osm_port_info_rcv.c b/opensm/opensm/osm_port_info_rcv.c index 90eb48b..a820069 100644 --- a/opensm/opensm/osm_port_info_rcv.c +++ b/opensm/opensm/osm_port_info_rcv.c @@ -72,9 +72,8 @@ __osm_pi_rcv_set_sm(IN osm_sm_t * sm, OSM_LOG_ENTER(sm->p_log); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Setting IS_SM bit in port attributes\n"); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Setting IS_SM bit in port attributes\n"); p_dr_path = osm_physp_get_dr_path_ptr(p_physp); h_bind = osm_dr_path_get_bind_handle(p_dr_path); @@ -165,12 +164,10 @@ __osm_pi_rcv_process_endport(IN osm_sm_t * sm, "Ignoring SM on port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); else { - if (osm_log_is_active - (sm->p_log, OSM_LOG_VERBOSE)) - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Detected another SM. Requesting SMInfo" - "\n\t\t\t\tPort 0x%" PRIx64 - "\n", cl_ntoh64(port_guid)); + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Detected another SM. Requesting SMInfo" + "\n\t\t\t\tPort 0x%" PRIx64 "\n", + cl_ntoh64(port_guid)); /* This port indicates it's an SM and it's not our own port. @@ -294,7 +291,6 @@ __osm_pi_rcv_process_switch_port(IN osm_sm_t * sm, "Failure initiating NodeInfo request (%s)\n", ib_get_err_str(status)); } else - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) OSM_LOG(sm->p_log, OSM_LOG_DEBUG, "Skipping SMP responder port 0x%X\n", p_pi->local_port_num); @@ -490,16 +486,14 @@ osm_pi_rcv_process_set(IN osm_sm_t * sm, IN osm_node_t * const p_node, port_guid, port_num, p_pi, level); } - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Received logical SetResp() for GUID 0x%" PRIx64 - ", port num 0x%X" - "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 - " TID 0x%" PRIx64 "\n", - cl_ntoh64(port_guid), - port_num, - cl_ntoh64(osm_node_get_node_guid(p_node)), - cl_ntoh64(p_smp->trans_id)); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Received logical SetResp() for GUID 0x%" PRIx64 + ", port num 0x%X" + "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 + " TID 0x%" PRIx64 "\n", + cl_ntoh64(port_guid), port_num, + cl_ntoh64(osm_node_get_node_guid(p_node)), + cl_ntoh64(p_smp->trans_id)); osm_physp_set_port_info(p_physp, p_pi); @@ -603,15 +597,12 @@ void osm_pi_rcv_process(IN void *context, IN void *data) This PortInfo arrived because we did a Get() method, most likely due to a subnet sweep in progress. */ - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Discovered port num 0x%X with GUID 0x%" PRIx64 - " for parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - port_num, - cl_ntoh64(port_guid), - cl_ntoh64(node_guid), - cl_ntoh64(p_smp->trans_id)); + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Discovered port num 0x%X with GUID 0x%" PRIx64 + " for parent node GUID 0x%" PRIx64 + ", TID 0x%" PRIx64 "\n", + port_num, cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); p_physp = osm_node_get_physp_ptr(p_node, port_num); @@ -621,10 +612,8 @@ void osm_pi_rcv_process(IN void *context, IN void *data) continue processing as normal. */ if (!p_physp) { - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Initializing port number 0x%X\n", - port_num); + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Initializing port number 0x%X\n", port_num); p_physp = &p_node->physp_table[port_num]; osm_physp_init(p_physp, port_guid, diff --git a/opensm/opensm/osm_req.c b/opensm/opensm/osm_req.c index 5454144..46f0d9d 100644 --- a/opensm/opensm/osm_req.c +++ b/opensm/opensm/osm_req.c @@ -96,13 +96,10 @@ osm_req_get(IN osm_sm_t * sm, tid = cl_hton64((uint64_t) cl_atomic_inc(&sm->sm_trans_id)); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Getting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", - ib_get_sm_attr_str(attr_id), - cl_ntoh16(attr_id), - cl_ntoh32(attr_mod), cl_ntoh64(tid)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Getting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", + ib_get_sm_attr_str(attr_id), cl_ntoh16(attr_id), + cl_ntoh32(attr_mod), cl_ntoh64(tid)); ib_smp_init_new(osm_madw_get_smp_ptr(p_madw), IB_MAD_METHOD_GET, @@ -177,13 +174,10 @@ osm_req_set(IN osm_sm_t * sm, tid = cl_hton64((uint64_t) cl_atomic_inc(&sm->sm_trans_id)); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Setting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", - ib_get_sm_attr_str(attr_id), - cl_ntoh16(attr_id), - cl_ntoh32(attr_mod), cl_ntoh64(tid)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Setting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", + ib_get_sm_attr_str(attr_id), cl_ntoh16(attr_id), + cl_ntoh32(attr_mod), cl_ntoh64(tid)); ib_smp_init_new(osm_madw_get_smp_ptr(p_madw), IB_MAD_METHOD_SET, diff --git a/opensm/opensm/osm_resp.c b/opensm/opensm/osm_resp.c index d141812..eb544db 100644 --- a/opensm/opensm/osm_resp.c +++ b/opensm/opensm/osm_resp.c @@ -144,13 +144,10 @@ osm_resp_send(IN osm_sm_t * sm, p_madw->resp_expected = FALSE; p_madw->fail_msg = CL_DISP_MSGID_NONE; - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, "Responding to %s (0x%X)" - "\n\t\t\t\tattribute modifier 0x%X, TID 0x%" PRIx64 - "\n", ib_get_sm_attr_str(p_smp->attr_id), - cl_ntoh16(p_smp->attr_id), cl_ntoh32(p_smp->attr_mod), - cl_ntoh64(p_smp->trans_id)); - } + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, "Responding to %s (0x%X)" + "\n\t\t\t\tattribute modifier 0x%X, TID 0x%" PRIx64 "\n", + ib_get_sm_attr_str(p_smp->attr_id), cl_ntoh16(p_smp->attr_id), + cl_ntoh32(p_smp->attr_mod), cl_ntoh64(p_smp->trans_id)); osm_vl15_post(sm->p_vl15, p_madw); diff --git a/opensm/opensm/osm_sa_guidinfo_record.c b/opensm/opensm/osm_sa_guidinfo_record.c index aaf4d6c..1848bc1 100644 --- a/opensm/opensm/osm_sa_guidinfo_record.c +++ b/opensm/opensm/osm_sa_guidinfo_record.c @@ -94,11 +94,9 @@ __osm_gir_rcv_new_gir(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New GUIDInfoRecord: lid 0x%X, block num %d\n", - cl_ntoh16(match_lid), block_num); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New GUIDInfoRecord: lid 0x%X, block num %d\n", + cl_ntoh16(match_lid), block_num); memset(p_rec_item, 0, sizeof(*p_rec_item)); @@ -138,13 +136,9 @@ __osm_sa_gir_create_gir(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Looking for GUIDRecord with LID: %u GUID:0x%016" - PRIx64 "\n", cl_ntoh16(match_lid), - cl_ntoh64(match_port_guid) - ); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Looking for GUIDRecord with LID: %u GUID:0x%016" + PRIx64 "\n", cl_ntoh16(match_lid), cl_ntoh64(match_port_guid)); /* For switches, do not return the GUIDInfo record(s) @@ -201,11 +195,9 @@ __osm_sa_gir_create_gir(IN osm_sa_t * sa, /* We validate that the lid belongs to this node. */ - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Comparing LID: %u <= %u <= %u\n", - base_lid_ho, match_lid_ho, max_lid_ho); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Comparing LID: %u <= %u <= %u\n", + base_lid_ho, match_lid_ho, max_lid_ho); if (match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho) diff --git a/opensm/opensm/osm_sa_informinfo.c b/opensm/opensm/osm_sa_informinfo.c index 334d22e..c8bfad8 100644 --- a/opensm/opensm/osm_sa_informinfo.c +++ b/opensm/opensm/osm_sa_informinfo.c @@ -221,10 +221,8 @@ __osm_infr_rcv_respond(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Generating successful InformInfo response\n"); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Generating successful InformInfo response\n"); item = malloc(sizeof(*item)); if (!item) { diff --git a/opensm/opensm/osm_sa_lft_record.c b/opensm/opensm/osm_sa_lft_record.c index e69211a..9589cac 100644 --- a/opensm/opensm/osm_sa_lft_record.c +++ b/opensm/opensm/osm_sa_lft_record.c @@ -88,12 +88,11 @@ __osm_lftr_rcv_new_lftr(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New LinearForwardingTable: sw 0x%016" PRIx64 - "\n\t\t\t\tblock 0x%02X lid 0x%02X\n", - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), - block, cl_ntoh16(lid)); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New LinearForwardingTable: sw 0x%016" PRIx64 + "\n\t\t\t\tblock 0x%02X lid 0x%02X\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), + block, cl_ntoh16(lid)); memset(p_rec_item, 0, sizeof(*p_rec_item)); diff --git a/opensm/opensm/osm_sa_link_record.c b/opensm/opensm/osm_sa_link_record.c index f08eee1..84dad6b 100644 --- a/opensm/opensm/osm_sa_link_record.c +++ b/opensm/opensm/osm_sa_link_record.c @@ -193,14 +193,12 @@ __osm_lr_rcv_get_physp_link(IN osm_sa_t * sa, if (to_base_lid != (p_lr->to_lid & lmc_mask)) goto Exit; - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Acquiring link record\n" - "\t\t\t\tsrc port 0x%" PRIx64 " (port 0x%X)" - ", dest port 0x%" PRIx64 " (port 0x%X)\n", - cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), - src_port_num, - cl_ntoh64(osm_physp_get_port_guid(p_dest_physp)), - dest_port_num); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Acquiring link record\n" + "\t\t\t\tsrc port 0x%" PRIx64 " (port 0x%X)" + ", dest port 0x%" PRIx64 " (port 0x%X)\n", + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), src_port_num, + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp)), + dest_port_num); __osm_lr_rcv_build_physp_link(sa, from_base_lid, to_base_lid, diff --git a/opensm/opensm/osm_sa_mad_ctrl.c b/opensm/opensm/osm_sa_mad_ctrl.c index 08026bc..f2ce125 100644 --- a/opensm/opensm/osm_sa_mad_ctrl.c +++ b/opensm/opensm/osm_sa_mad_ctrl.c @@ -296,10 +296,8 @@ __osm_sa_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, */ cl_atomic_inc(&p_ctrl->p_stats->sa_mads_rcvd); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "%u SA MADs received\n", p_ctrl->p_stats->sa_mads_rcvd); - } + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, + "%u SA MADs received\n", p_ctrl->p_stats->sa_mads_rcvd); /* * C15-0.1.3 requires not responding to any MAD if the SM is @@ -430,12 +428,9 @@ __osm_sa_mad_ctrl_send_err_callback(IN void *bind_context, /* __osm_sm_mad_ctrl_update_wire_stats( p_ctrl ); */ if (osm_madw_get_err_msg(p_madw) != CL_DISP_MSGID_NONE) { - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(osm_madw_get_err_msg - (p_madw))); - } + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(osm_madw_get_err_msg(p_madw))); status = cl_disp_post(p_ctrl->h_disp, osm_madw_get_err_msg(p_madw), diff --git a/opensm/opensm/osm_sa_mft_record.c b/opensm/opensm/osm_sa_mft_record.c index 8e04705..06d2538 100644 --- a/opensm/opensm/osm_sa_mft_record.c +++ b/opensm/opensm/osm_sa_mft_record.c @@ -90,12 +90,11 @@ __osm_mftr_rcv_new_mftr(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New MulticastForwardingTable: sw 0x%016" PRIx64 - "\n\t\t\t\tblock %u position %u lid 0x%02X\n", - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), - block, position, cl_ntoh16(lid)); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New MulticastForwardingTable: sw 0x%016" PRIx64 + "\n\t\t\t\tblock %u position %u lid 0x%02X\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), + block, position, cl_ntoh16(lid)); position_block_num = ((uint16_t) position << 12) | (block & IB_MCAST_BLOCK_ID_MASK_HO); diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c index 86a1d50..2b8e00a 100644 --- a/opensm/opensm/osm_sa_multipath_record.c +++ b/opensm/opensm/osm_sa_multipath_record.c @@ -258,9 +258,8 @@ __osm_mpr_rcv_get_path_parms(IN osm_sa_t * sa, valid_sl_mask &= ~(1 << i); } if (!valid_sl_mask) { - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "All the SLs lead to VL15 on this path\n"); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "All the SLs lead to VL15 on this path\n"); status = IB_NOT_FOUND; goto Exit; } @@ -372,10 +371,9 @@ __osm_mpr_rcv_get_path_parms(IN osm_sa_t * sa, valid_sl_mask &= ~(1 << i); } if (!valid_sl_mask) { - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "All the SLs lead to VL15 " - "on this path\n"); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "All the SLs lead to VL15 " + "on this path\n"); status = IB_NOT_FOUND; goto Exit; } @@ -393,10 +391,8 @@ __osm_mpr_rcv_get_path_parms(IN osm_sa_t * sa, if (rate > ib_port_info_compute_rate(p_pi)) rate = ib_port_info_compute_rate(p_pi); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Path min MTU = %u, min rate = %u\n", mtu, rate); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Path min MTU = %u, min rate = %u\n", mtu, rate); /* * Get QoS Level object according to the MultiPath request @@ -409,12 +405,10 @@ __osm_mpr_rcv_get_path_parms(IN osm_sa_t * sa, p_mpr, p_src_physp, p_dest_physp, comp_mask))) { - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "MultiPathRecord request matches QoS Level '%s' (%s)\n", - p_qos_level->name, - (p_qos_level->use) ? p_qos_level-> - use : "no description"); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "MultiPathRecord request matches QoS Level '%s' (%s)\n", + p_qos_level->name, + p_qos_level->use ? p_qos_level->use : "no description"); if (p_qos_level->mtu_limit_set && (mtu > p_qos_level->mtu_limit)) @@ -737,11 +731,10 @@ __osm_mpr_rcv_get_path_parms(IN osm_sa_t * sa, p_parms->sl = required_sl; p_parms->hops = hops; - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "MultiPath params:" - " mtu = %u, rate = %u, packet lifetime = %u," - " pkey = 0x%04X, sl = %u, hops = %u\n", mtu, rate, - pkt_life, cl_ntoh16(required_pkey), required_sl, hops); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "MultiPath params:" + " mtu = %u, rate = %u, packet lifetime = %u," + " pkey = 0x%04X, sl = %u, hops = %u\n", mtu, rate, + pkt_life, cl_ntoh16(required_pkey), required_sl, hops); Exit: OSM_LOG_EXIT(sa->p_log); @@ -820,10 +813,8 @@ __osm_mpr_rcv_get_lid_pair_path(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src LID %u, Dest LID %u\n", - src_lid_ho, dest_lid_ho); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Src LID %u, Dest LID %u\n", + src_lid_ho, dest_lid_ho); p_pr_item = malloc(sizeof(*p_pr_item)); if (p_pr_item == NULL) { @@ -906,12 +897,10 @@ __osm_mpr_rcv_get_port_pair_paths(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src port 0x%016" PRIx64 ", " - "Dst port 0x%016" PRIx64 "\n", - cl_ntoh64(osm_port_get_guid(p_src_port)), - cl_ntoh64(osm_port_get_guid(p_dest_port))); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Src port 0x%016" PRIx64 ", Dst port 0x%016" PRIx64 "\n", + cl_ntoh64(osm_port_get_guid(p_src_port)), + cl_ntoh64(osm_port_get_guid(p_dest_port))); /* Check that the req_port, src_port and dest_port all share a pkey. The check is done on the default physical port of the ports. */ @@ -972,11 +961,9 @@ __osm_mpr_rcv_get_port_pair_paths(IN osm_sa_t * sa, osm_port_get_lid_range_ho(p_dest_port, &dest_lid_min_ho, &dest_lid_max_ho); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src LID [%u-%u], Dest LID [%u-%u]\n", - src_lid_min_ho, src_lid_max_ho, - dest_lid_min_ho, dest_lid_max_ho); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Src LID [%u-%u], Dest LID [%u-%u]\n", + src_lid_min_ho, src_lid_max_ho, + dest_lid_min_ho, dest_lid_max_ho); src_lid_ho = src_lid_min_ho; dest_lid_ho = dest_lid_min_ho; @@ -1101,12 +1088,10 @@ __osm_mpr_rcv_get_apm_port_pair_paths(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src port 0x%016" PRIx64 ", " - "Dst port 0x%016" PRIx64 ", base offs %d\n", - cl_ntoh64(osm_port_get_guid(p_src_port)), - cl_ntoh64(osm_port_get_guid(p_dest_port)), base_offs); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Src port 0x%016" PRIx64 ", " + "Dst port 0x%016" PRIx64 ", base offs %d\n", + cl_ntoh64(osm_port_get_guid(p_src_port)), + cl_ntoh64(osm_port_get_guid(p_dest_port)), base_offs); osm_port_get_lid_range_ho(p_src_port, &src_lid_min_ho, &src_lid_max_ho); osm_port_get_lid_range_ho(p_dest_port, &dest_lid_min_ho, diff --git a/opensm/opensm/osm_sa_node_record.c b/opensm/opensm/osm_sa_node_record.c index 1ba418e..a18d0ef 100644 --- a/opensm/opensm/osm_sa_node_record.c +++ b/opensm/opensm/osm_sa_node_record.c @@ -88,13 +88,11 @@ __osm_nr_rcv_new_nr(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New NodeRecord: node 0x%016" PRIx64 - "\n\t\t\t\tport 0x%016" PRIx64 ", lid 0x%X\n", - cl_ntoh64(osm_node_get_node_guid(p_node)), - cl_ntoh64(port_guid), cl_ntoh16(lid) - ); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New NodeRecord: node 0x%016" PRIx64 + "\n\t\t\t\tport 0x%016" PRIx64 ", lid 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + cl_ntoh64(port_guid), cl_ntoh16(lid)); memset(p_rec_item, 0, sizeof(*p_rec_item)); @@ -133,13 +131,10 @@ __osm_nr_rcv_create_nr(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Looking for NodeRecord with LID: %u GUID:0x%016" - PRIx64 "\n", cl_ntoh16(match_lid), - cl_ntoh64(match_port_guid) - ); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Looking for NodeRecord with LID: %u GUID:0x%016" + PRIx64 "\n", cl_ntoh16(match_lid), + cl_ntoh64(match_port_guid)); /* For switches, do not return the NodeInfo record @@ -175,11 +170,9 @@ __osm_nr_rcv_create_nr(IN osm_sa_t * sa, /* We validate that the lid belongs to this node. */ - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Comparing LID: %u <= %u <= %u\n", - base_lid_ho, match_lid_ho, max_lid_ho); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Comparing LID: %u <= %u <= %u\n", + base_lid_ho, match_lid_ho, max_lid_ho); if (match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho) @@ -217,20 +210,12 @@ __osm_nr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, IN void *context) match_lid = p_rcvd_rec->lid; if (comp_mask & IB_NR_COMPMASK_NODEGUID) { - /* - DEBUG TOP - */ - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Looking for node 0x%016" PRIx64 - ", found 0x%016" PRIx64 "\n", - cl_ntoh64(p_rcvd_rec->node_info.node_guid), - cl_ntoh64(osm_node_get_node_guid(p_node)) - ); - } - /* - DEBUG BOTTOM - */ + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Looking for node 0x%016" PRIx64 + ", found 0x%016" PRIx64 "\n", + cl_ntoh64(p_rcvd_rec->node_info.node_guid), + cl_ntoh64(osm_node_get_node_guid(p_node))); + if ((p_node->node_info.node_guid != p_rcvd_rec->node_info.node_guid)) goto Exit; diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index 2cdce03..d60dc01 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -266,9 +266,8 @@ __osm_pr_rcv_get_path_parms(IN osm_sa_t * sa, valid_sl_mask &= ~(1 << i); } if (!valid_sl_mask) { - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "All the SLs lead to VL15 on this path\n"); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "All the SLs lead to VL15 on this path\n"); status = IB_NOT_FOUND; goto Exit; } @@ -379,10 +378,8 @@ __osm_pr_rcv_get_path_parms(IN osm_sa_t * sa, valid_sl_mask &= ~(1 << i); } if (!valid_sl_mask) { - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "All the SLs lead to VL15 " - "on this path\n"); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "All the SLs " + "lead to VL15 on this path\n"); status = IB_NOT_FOUND; goto Exit; } @@ -400,9 +397,8 @@ __osm_pr_rcv_get_path_parms(IN osm_sa_t * sa, if (rate > ib_port_info_compute_rate(p_pi)) rate = ib_port_info_compute_rate(p_pi); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Path min MTU = %u, min rate = %u\n", mtu, rate); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Path min MTU = %u, min rate = %u\n", mtu, rate); /* * Get QoS Level object according to the path request @@ -414,12 +410,10 @@ __osm_pr_rcv_get_path_parms(IN osm_sa_t * sa, osm_qos_policy_get_qos_level_by_pr(sa->p_subn->p_qos_policy, p_pr, p_src_physp, p_dest_physp, comp_mask))) { - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "PathRecord request matches QoS Level '%s' (%s)\n", - p_qos_level->name, - (p_qos_level->use) ? p_qos_level-> - use : "no description"); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "PathRecord request matches QoS Level '%s' (%s)\n", + p_qos_level->name, p_qos_level->use ? + p_qos_level->use : "no description"); if (p_qos_level->mtu_limit_set && (mtu > p_qos_level->mtu_limit)) @@ -745,11 +739,9 @@ __osm_pr_rcv_get_path_parms(IN osm_sa_t * sa, p_parms->pkey = pkey; p_parms->sl = sl; - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Path params:" - " mtu = %u, rate = %u, packet lifetime = %u," - " pkey = 0x%04X, sl = %u\n", - mtu, rate, pkt_life, cl_ntoh16(pkey), sl); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Path params: mtu = %u, rate = %u," + " packet lifetime = %u, pkey = 0x%04X, sl = %u\n", + mtu, rate, pkt_life, cl_ntoh16(pkey), sl); Exit: OSM_LOG_EXIT(sa->p_log); return (status); @@ -846,10 +838,8 @@ __osm_pr_rcv_get_lid_pair_path(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src LID %u, Dest LID %u\n", - src_lid_ho, dest_lid_ho); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Src LID %u, Dest LID %u\n", + src_lid_ho, dest_lid_ho); p_pr_item = malloc(sizeof(*p_pr_item)); if (p_pr_item == NULL) { @@ -931,12 +921,10 @@ __osm_pr_rcv_get_port_pair_paths(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src port 0x%016" PRIx64 ", " - "Dst port 0x%016" PRIx64 "\n", - cl_ntoh64(osm_port_get_guid(p_src_port)), - cl_ntoh64(osm_port_get_guid(p_dest_port))); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Src port 0x%016" PRIx64 ", Dst port 0x%016" PRIx64 "\n", + cl_ntoh64(osm_port_get_guid(p_src_port)), + cl_ntoh64(osm_port_get_guid(p_dest_port))); /* Check that the req_port, src_port and dest_port all share a pkey. The check is done on the default physical port of the ports. */ @@ -1025,11 +1013,10 @@ __osm_pr_rcv_get_port_pair_paths(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Src LIDs [%u-%u], Dest LIDs [%u-%u]\n", - src_lid_min_ho, src_lid_max_ho, - dest_lid_min_ho, dest_lid_max_ho); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Src LIDs [%u-%u], Dest LIDs [%u-%u]\n", + src_lid_min_ho, src_lid_max_ho, + dest_lid_min_ho, dest_lid_max_ho); src_lid_ho = src_lid_min_ho; dest_lid_ho = dest_lid_min_ho; diff --git a/opensm/opensm/osm_sa_pkey_record.c b/opensm/opensm/osm_sa_pkey_record.c index db341f1..f7f933b 100644 --- a/opensm/opensm/osm_sa_pkey_record.c +++ b/opensm/opensm/osm_sa_pkey_record.c @@ -91,13 +91,11 @@ __osm_sa_pkey_create(IN osm_sa_t * sa, else lid = osm_node_get_base_lid(p_physp->p_node, 0); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New P_Key table for: port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X Block:%u\n", - cl_ntoh64(osm_physp_get_port_guid(p_physp)), - cl_ntoh16(lid), osm_physp_get_port_num(p_physp), block); - } + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New P_Key table for: port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X Block:%u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp), block); memset(p_rec_item, 0, sizeof(*p_rec_item)); diff --git a/opensm/opensm/osm_sa_portinfo_record.c b/opensm/opensm/osm_sa_portinfo_record.c index 267fe4a..afe61e1 100644 --- a/opensm/opensm/osm_sa_portinfo_record.c +++ b/opensm/opensm/osm_sa_portinfo_record.c @@ -92,12 +92,11 @@ __osm_pir_rcv_new_pir(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New PortInfoRecord: port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X\n", - cl_ntoh64(osm_physp_get_port_guid(p_physp)), - cl_ntoh16(lid), osm_physp_get_port_num(p_physp)); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New PortInfoRecord: port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp)); memset(p_rec_item, 0, sizeof(*p_rec_item)); @@ -147,10 +146,9 @@ __osm_sa_pir_create(IN osm_sa_t * sa, /* We validate that the lid belongs to this node. */ - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Comparing LID: %u <= %u <= %u\n", - base_lid_ho, match_lid_ho, max_lid_ho); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Comparing LID: %u <= %u <= %u\n", + base_lid_ho, match_lid_ho, max_lid_ho); if (match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho) goto Exit; diff --git a/opensm/opensm/osm_sa_service_record.c b/opensm/opensm/osm_sa_service_record.c index c9f4ee0..93d86b5 100644 --- a/opensm/opensm/osm_sa_service_record.c +++ b/opensm/opensm/osm_sa_service_record.c @@ -487,10 +487,9 @@ osm_sr_rcv_process_get_method(IN osm_sa_t * sa, p_recvd_service_rec = (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - osm_dump_service_record(sa->p_log, - p_recvd_service_rec, OSM_LOG_DEBUG); - } + if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) + osm_dump_service_record(sa->p_log, p_recvd_service_rec, + OSM_LOG_DEBUG); cl_qlist_init(&sr_match_item.sr_list); sr_match_item.p_service_rec = p_recvd_service_rec; @@ -547,10 +546,9 @@ osm_sr_rcv_process_set_method(IN osm_sa_t * sa, comp_mask = p_sa_mad->comp_mask; - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - osm_dump_service_record(sa->p_log, - p_recvd_service_rec, OSM_LOG_DEBUG); - } + if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) + osm_dump_service_record(sa->p_log, p_recvd_service_rec, + OSM_LOG_DEBUG); if ((comp_mask & (IB_SR_COMPMASK_SID | IB_SR_COMPMASK_SGID)) != (IB_SR_COMPMASK_SID | IB_SR_COMPMASK_SGID)) { @@ -657,10 +655,9 @@ osm_sr_rcv_process_delete_method(IN osm_sa_t * sa, comp_mask = p_sa_mad->comp_mask; - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - osm_dump_service_record(sa->p_log, - p_recvd_service_rec, OSM_LOG_DEBUG); - } + if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) + osm_dump_service_record(sa->p_log, p_recvd_service_rec, + OSM_LOG_DEBUG); /* Grab the lock */ cl_plock_excl_acquire(sa->p_lock); diff --git a/opensm/opensm/osm_sa_slvl_record.c b/opensm/opensm/osm_sa_slvl_record.c index 0b33307..46b66a6 100644 --- a/opensm/opensm/osm_sa_slvl_record.c +++ b/opensm/opensm/osm_sa_slvl_record.c @@ -98,13 +98,11 @@ __osm_sa_slvl_create(IN osm_sa_t * sa, else lid = osm_node_get_base_lid(p_physp->p_node, 0); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New SLtoVL Map for: OUT port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X to In Port:%u\n", - cl_ntoh64(osm_physp_get_port_guid(p_physp)), - cl_ntoh16(lid), osm_physp_get_port_num(p_physp), - in_port_idx); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New SLtoVL Map for: OUT port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X to In Port:%u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp), in_port_idx); memset(p_rec_item, 0, sizeof(*p_rec_item)); diff --git a/opensm/opensm/osm_sa_sminfo_record.c b/opensm/opensm/osm_sa_sminfo_record.c index f23c3e0..e736c0c 100644 --- a/opensm/opensm/osm_sa_sminfo_record.c +++ b/opensm/opensm/osm_sa_sminfo_record.c @@ -98,9 +98,8 @@ __osm_smir_rcv_new_smir(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New SMInfo: GUID 0x%016" PRIx64 "\n", cl_ntoh64(guid)); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New SMInfo: GUID 0x%016" PRIx64 "\n", cl_ntoh64(guid)); memset(p_rec_item, 0, sizeof(*p_rec_item)); diff --git a/opensm/opensm/osm_sa_sw_info_record.c b/opensm/opensm/osm_sa_sw_info_record.c index 9a3c112..7dd3de7 100644 --- a/opensm/opensm/osm_sa_sw_info_record.c +++ b/opensm/opensm/osm_sa_sw_info_record.c @@ -87,9 +87,8 @@ __osm_sir_rcv_new_sir(IN osm_sa_t * sa, goto Exit; } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New SwitchInfoRecord: lid 0x%X\n", cl_ntoh16(lid)); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New SwitchInfoRecord: lid 0x%X\n", cl_ntoh16(lid)); memset(p_rec_item, 0, sizeof(*p_rec_item)); @@ -120,10 +119,9 @@ __osm_sir_rcv_create_sir(IN osm_sa_t * sa, OSM_LOG_ENTER(sa->p_log); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Looking for SwitchInfoRecord with LID: %u\n", - cl_ntoh16(match_lid)); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Looking for SwitchInfoRecord with LID: %u\n", + cl_ntoh16(match_lid)); /* In switches, the port guid is the node guid. */ p_port = @@ -156,10 +154,9 @@ __osm_sir_rcv_create_sir(IN osm_sa_t * sa, /* We validate that the lid belongs to this switch. */ - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Comparing LID: %u <= %u <= %u\n", - min_lid_ho, match_lid_ho, max_lid_ho); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Comparing LID: %u <= %u <= %u\n", + min_lid_ho, match_lid_ho, max_lid_ho); if (match_lid_ho < min_lid_ho || match_lid_ho > max_lid_ho) goto Exit; diff --git a/opensm/opensm/osm_sa_vlarb_record.c b/opensm/opensm/osm_sa_vlarb_record.c index eb4dd81..2787fc5 100644 --- a/opensm/opensm/osm_sa_vlarb_record.c +++ b/opensm/opensm/osm_sa_vlarb_record.c @@ -98,12 +98,11 @@ __osm_sa_vl_arb_create(IN osm_sa_t * sa, else lid = osm_node_get_base_lid(p_physp->p_node, 0); - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "New VLArbitration for: port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X Block:%u\n", - cl_ntoh64(osm_physp_get_port_guid(p_physp)), - cl_ntoh16(lid), osm_physp_get_port_num(p_physp), block); + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "New VLArbitration for: port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X Block:%u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp), block); memset(p_rec_item, 0, sizeof(*p_rec_item)); diff --git a/opensm/opensm/osm_slvl_map_rcv.c b/opensm/opensm/osm_slvl_map_rcv.c index 9899625..000c2a6 100644 --- a/opensm/opensm/osm_slvl_map_rcv.c +++ b/opensm/opensm/osm_slvl_map_rcv.c @@ -123,14 +123,11 @@ void osm_slvl_rcv_process(IN void *context, IN void *p_data) We do not mind if this is a result of a set or get - all we want is to update the subnet. */ - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Got SLtoVL get response in_port_num %u out_port_num %u with GUID 0x%" - PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" - PRIx64 "\n", in_port_num, out_port_num, - cl_ntoh64(port_guid), cl_ntoh64(node_guid), - cl_ntoh64(p_smp->trans_id)); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Got SLtoVL get response in_port_num %u out_port_num %u with " + "GUID 0x%" PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" + PRIx64 "\n", in_port_num, out_port_num, cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); /* Determine if we encountered a new Physical Port. diff --git a/opensm/opensm/osm_sm_mad_ctrl.c b/opensm/opensm/osm_sm_mad_ctrl.c index aa91c21..2815aa2 100644 --- a/opensm/opensm/osm_sm_mad_ctrl.c +++ b/opensm/opensm/osm_sm_mad_ctrl.c @@ -76,19 +76,16 @@ __osm_sm_mad_ctrl_retire_trans_mad(IN osm_sm_mad_ctrl_t * const p_ctrl, /* Return the MAD & wrapper to the pool. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Retiring MAD with TID 0x%" PRIx64 "\n", - cl_ntoh64(osm_madw_get_smp_ptr(p_madw)->trans_id)); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, + "Retiring MAD with TID 0x%" PRIx64 "\n", + cl_ntoh64(osm_madw_get_smp_ptr(p_madw)->trans_id)); osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); outstanding = cl_atomic_dec(&p_ctrl->p_stats->qp0_mads_outstanding); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "%u QP0 MADs outstanding\n", - p_ctrl->p_stats->qp0_mads_outstanding); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, "%u QP0 MADs outstanding\n", + p_ctrl->p_stats->qp0_mads_outstanding); if (outstanding == 0) { /* @@ -170,10 +167,9 @@ __osm_sm_mad_ctrl_update_wire_stats(IN osm_sm_mad_ctrl_t * const p_ctrl) mads_on_wire = cl_atomic_dec(&p_ctrl->p_stats->qp0_mads_outstanding_on_wire); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "%u SMPs on the wire, %u outstanding\n", mads_on_wire, - p_ctrl->p_stats->qp0_mads_outstanding); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, + "%u SMPs on the wire, %u outstanding\n", mads_on_wire, + p_ctrl->p_stats->qp0_mads_outstanding); /* We can signal the VL15 controller to send another MAD @@ -287,10 +283,8 @@ __osm_sm_mad_ctrl_process_get_resp(IN osm_sm_mad_ctrl_t * const p_ctrl, processing by the appropriate controller. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); @@ -358,10 +352,8 @@ __osm_sm_mad_ctrl_process_get(IN osm_sm_mad_ctrl_t * const p_ctrl, processing by the appropriate controller. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); @@ -440,10 +432,8 @@ __osm_sm_mad_ctrl_process_set(IN osm_sm_mad_ctrl_t * const p_ctrl, processing by the appropriate controller. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); @@ -531,10 +521,8 @@ __osm_sm_mad_ctrl_process_trap(IN osm_sm_mad_ctrl_t * const p_ctrl, processing by the appropriate controller. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); @@ -587,10 +575,8 @@ __osm_sm_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, */ cl_atomic_inc(&p_ctrl->p_stats->qp0_mads_rcvd); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "%u QP0 MADs received\n", - p_ctrl->p_stats->qp0_mads_rcvd); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, "%u QP0 MADs received\n", + p_ctrl->p_stats->qp0_mads_rcvd); p_smp = osm_madw_get_smp_ptr(p_madw); @@ -755,11 +741,9 @@ __osm_sm_mad_ctrl_send_err_cb(IN void *bind_context, IN osm_madw_t * p_madw) __osm_sm_mad_ctrl_update_wire_stats(p_ctrl); if (osm_madw_get_err_msg(p_madw) != CL_DISP_MSGID_NONE) { - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(osm_madw_get_err_msg - (p_madw))); + OSM_LOG(p_ctrl->p_log, OSM_LOG_DEBUG, + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(osm_madw_get_err_msg(p_madw))); status = cl_disp_post(p_ctrl->h_disp, osm_madw_get_err_msg(p_madw), diff --git a/opensm/opensm/osm_sm_state_mgr.c b/opensm/opensm/osm_sm_state_mgr.c index 6edba50..5736b8c 100644 --- a/opensm/opensm/osm_sm_state_mgr.c +++ b/opensm/opensm/osm_sm_state_mgr.c @@ -257,11 +257,10 @@ ib_api_status_t osm_sm_state_mgr_process(osm_sm_t * sm, */ cl_spinlock_acquire(&sm->state_lock); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Received signal %s in state %s\n", - osm_get_sm_mgr_signal_str(signal), - osm_get_sm_mgr_state_str(sm->p_subn->sm_state)); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Received signal %s in state %s\n", + osm_get_sm_mgr_signal_str(signal), + osm_get_sm_mgr_state_str(sm->p_subn->sm_state)); switch (sm->p_subn->sm_state) { case IB_SMINFO_STATE_DISCOVERING: @@ -476,11 +475,10 @@ ib_api_status_t osm_sm_state_mgr_check_legality(osm_sm_t * sm, */ cl_spinlock_acquire(&sm->state_lock); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Received signal %s in state %s\n", - osm_get_sm_mgr_signal_str(signal), - osm_get_sm_mgr_state_str(sm->p_subn->sm_state)); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Received signal %s in state %s\n", + osm_get_sm_mgr_signal_str(signal), + osm_get_sm_mgr_state_str(sm->p_subn->sm_state)); switch (sm->p_subn->sm_state) { case IB_SMINFO_STATE_DISCOVERING: diff --git a/opensm/opensm/osm_sminfo_rcv.c b/opensm/opensm/osm_sminfo_rcv.c index 9a262d2..22a9566 100644 --- a/opensm/opensm/osm_sminfo_rcv.c +++ b/opensm/opensm/osm_sminfo_rcv.c @@ -326,10 +326,9 @@ __osm_sminfo_rcv_process_get_sm(IN osm_sm_t * sm, p_smi = &p_sm->smi; - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Detected SM 0x%016" PRIx64 " in state %u\n", - cl_ntoh64(p_smi->guid), ib_sminfo_get_state(p_smi)); + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Detected SM 0x%016" PRIx64 " in state %u\n", + cl_ntoh64(p_smi->guid), ib_sminfo_get_state(p_smi)); /* Check the state of this SM vs. our own. diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index b9fd7f7..62b3914 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -93,14 +93,6 @@ static void __osm_state_mgr_reset_node_count(IN cl_map_item_t * const p_map_item, IN void *context) { osm_node_t *p_node = (osm_node_t *) p_map_item; - osm_sm_t *sm = context; - - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Resetting discovery count for node 0x%" PRIx64 - "(%s)\n", cl_ntoh64(osm_node_get_node_guid(p_node)), - p_node->print_desc); - } p_node->discovery_count = 0; } @@ -111,15 +103,6 @@ static void __osm_state_mgr_reset_port_count(IN cl_map_item_t * const p_map_item, IN void *context) { osm_port_t *p_port = (osm_port_t *) p_map_item; - osm_sm_t *sm = context; - - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Resetting discovery count for port 0x%" PRIx64 - "(node %s)\n", cl_ntoh64(osm_port_get_guid(p_port)), - p_port->p_node ? p_port->p_node-> - print_desc : "UNKNOWN"); - } p_port->discovery_count = 0; } @@ -131,15 +114,6 @@ __osm_state_mgr_reset_switch_count(IN cl_map_item_t * const p_map_item, IN void *context) { osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - osm_sm_t *sm = context; - - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Resetting discovery count for switch 0x%" PRIx64 - " (%s)\n", - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), - p_sw->p_node->print_desc); - } p_sw->discovery_count = 0; p_sw->need_update = 1; @@ -1302,11 +1276,10 @@ void osm_state_mgr_process(IN osm_sm_t * sm, IN osm_signal_t signal) OSM_LOG_ENTER(sm->p_log); - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Received signal %s in state %s\n", - osm_get_sm_signal_str(signal), - osm_get_sm_mgr_state_str(sm->p_subn->sm_state)); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Received signal %s in state %s\n", + osm_get_sm_signal_str(signal), + osm_get_sm_mgr_state_str(sm->p_subn->sm_state)); switch (signal) { case OSM_SIGNAL_SWEEP: diff --git a/opensm/opensm/osm_sw_info_rcv.c b/opensm/opensm/osm_sw_info_rcv.c index 362d54f..f75be65 100644 --- a/opensm/opensm/osm_sw_info_rcv.c +++ b/opensm/opensm/osm_sw_info_rcv.c @@ -149,9 +149,8 @@ __osm_si_rcv_get_fwd_tbl(IN osm_sm_t * sm, IN osm_switch_t * const p_sw) p_dr_path = osm_physp_get_dr_path_ptr(p_physp); for (block_id_ho = 0; block_id_ho <= max_block_id_ho; block_id_ho++) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Retrieving FT block %u\n", block_id_ho); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Retrieving FT block %u\n", block_id_ho); status = osm_req_get(sm, p_dr_path, IB_MAD_ATTR_LIN_FWD_TBL, cl_hton32(block_id_ho), @@ -226,15 +225,12 @@ __osm_si_rcv_get_mcast_fwd_tbl(IN osm_sm_t * sm, IN osm_switch_t * const p_sw) p_dr_path = osm_physp_get_dr_path_ptr(p_physp); for (block_id_ho = 0; block_id_ho <= max_block_id_ho; block_id_ho++) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Retrieving MFT block %u\n", block_id_ho); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Retrieving MFT block %u\n", block_id_ho); for (position = 0; position <= max_position; position++) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Retrieving MFT position %u\n", - position); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Retrieving MFT position %u\n", position); attr_mod_ho = block_id_ho | position << IB_MCAST_POSITION_SHIFT; @@ -384,15 +380,13 @@ __osm_si_rcv_process_existing(IN osm_sm_t * sm, p_si_context = osm_madw_get_si_context_ptr(p_madw); if (p_si_context->set_method) { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Received logical SetResp()\n"); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Received logical SetResp()\n"); osm_switch_set_switch_info(p_sw, p_si); } else { - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Received logical GetResp()\n"); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Received logical GetResp()\n"); osm_switch_set_switch_info(p_sw, p_si); @@ -474,10 +468,9 @@ void osm_si_rcv_process(IN void *context, IN void *data) node_guid = p_context->node_guid; - if (osm_log_is_active(sm->p_log, OSM_LOG_DEBUG)) - OSM_LOG(sm->p_log, OSM_LOG_DEBUG, - "Switch GUID 0x%016" PRIx64 ", TID 0x%" PRIx64 "\n", - cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + OSM_LOG(sm->p_log, OSM_LOG_DEBUG, + "Switch GUID 0x%016" PRIx64 ", TID 0x%" PRIx64 "\n", + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); CL_PLOCK_EXCL_ACQUIRE(sm->p_lock); diff --git a/opensm/opensm/osm_ucast_mgr.c b/opensm/opensm/osm_ucast_mgr.c index 82d1f4b..a686dc9 100644 --- a/opensm/opensm/osm_ucast_mgr.c +++ b/opensm/opensm/osm_ucast_mgr.c @@ -150,14 +150,12 @@ __osm_ucast_mgr_process_neighbor(IN osm_ucast_mgr_t * const p_mgr, OSM_LOG_ENTER(p_mgr->p_log); - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Node 0x%" PRIx64 ", remote node 0x%" PRIx64 - ", port 0x%X, remote port 0x%X\n", - cl_ntoh64(osm_node_get_node_guid(p_this_sw->p_node)), - cl_ntoh64(osm_node_get_node_guid(p_remote_sw->p_node)), - port_num, remote_port_num); - } + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Node 0x%" PRIx64 ", remote node 0x%" PRIx64 + ", port 0x%X, remote port 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_this_sw->p_node)), + cl_ntoh64(osm_node_get_node_guid(p_remote_sw->p_node)), + port_num, remote_port_num); p_next_sw = (osm_switch_t *) cl_qmap_head(&p_mgr->p_subn->sw_guid_tbl); while (p_next_sw != @@ -244,12 +242,11 @@ __osm_ucast_mgr_process_port(IN osm_ucast_mgr_t * const p_mgr, /* ignore potential overflow - it is handled in osm_switch.c */ start_from = osm_switch_get_port_by_lid(p_sw, lid_ho - 1) + 1; - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Processing port 0x%" PRIx64 " (\'%s\' port %u), LID %u [%u,%u]\n", - cl_ntoh64(osm_port_get_guid(p_port)), - p_port->p_node->print_desc, p_port->p_physp->port_num, - lid_ho, min_lid_ho, max_lid_ho); + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Processing port 0x%" PRIx64 " (\'%s\' port %u), LID %u [%u,%u]\n", + cl_ntoh64(osm_port_get_guid(p_port)), + p_port->p_node->print_desc, p_port->p_physp->port_num, + lid_ho, min_lid_ho, max_lid_ho); /* TODO - This should be runtime error, not a CL_ASSERT() */ CL_ASSERT(max_lid_ho < osm_switch_get_fwd_tbl_size(p_sw)); @@ -382,11 +379,8 @@ osm_ucast_mgr_set_fwd_table(IN osm_ucast_mgr_t * const p_mgr, } if (set_swinfo_require) { - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Setting switch FT top to LID %u\n", - p_sw->max_lid_ho); - } + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Setting switch FT top to LID %u\n", p_sw->max_lid_ho); context.si_context.light_sweep = FALSE; context.si_context.node_guid = osm_node_get_node_guid(p_node); @@ -421,10 +415,8 @@ osm_ucast_mgr_set_fwd_table(IN osm_ucast_mgr_t * const p_mgr, !memcmp(block, p_mgr->lft_buf + block_id_ho * 64, 64)) continue; - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Writing FT block %u\n", block_id_ho); - } + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Writing FT block %u\n", block_id_ho); status = osm_req_set(p_mgr->sm, p_path, p_mgr->lft_buf + block_id_ho * 64, @@ -501,11 +493,9 @@ __osm_ucast_mgr_process_tbl(IN cl_map_item_t * const p_map_item, CL_ASSERT(p_sw && p_sw->p_node); - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Processing switch 0x%" PRIx64 "\n", - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node))); - } + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Processing switch 0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node))); /* Initialize LIDs in buffer to invalid port number. */ memset(p_mgr->lft_buf, OSM_NO_PATH, IB_LID_UCAST_END_HO + 1); @@ -558,11 +548,9 @@ __osm_ucast_mgr_process_neighbors(IN cl_map_item_t * const p_map_item, CL_ASSERT(p_node); CL_ASSERT(osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH); - if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "Processing switch with GUID 0x%" PRIx64 "\n", - cl_ntoh64(osm_node_get_node_guid(p_node))); - } + OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, + "Processing switch with GUID 0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); num_ports = osm_node_get_num_physp(p_node); diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c index a47f9ba..7f14e73 100644 --- a/opensm/opensm/osm_vl15intf.c +++ b/opensm/opensm/osm_vl15intf.c @@ -83,14 +83,13 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) p_madw, p_madw->resp_expected); if (status == IB_SUCCESS) { - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "%u QP0 MADs on wire, %u outstanding, " - "%u unicasts sent, %u total sent\n", - p_vl->p_stats->qp0_mads_outstanding_on_wire, - p_vl->p_stats->qp0_mads_outstanding, - p_vl->p_stats->qp0_unicasts_sent, - p_vl->p_stats->qp0_mads_sent); + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, + "%u QP0 MADs on wire, %u outstanding, " + "%u unicasts sent, %u total sent\n", + p_vl->p_stats->qp0_mads_outstanding_on_wire, + p_vl->p_stats->qp0_mads_outstanding, + p_vl->p_stats->qp0_unicasts_sent, + p_vl->p_stats->qp0_mads_sent); return; } @@ -143,10 +142,8 @@ static void __osm_vl15_poller(IN void *p_ptr) cl_spinlock_release(&p_vl->lock); if (p_madw != (osm_madw_t *) cl_qlist_end(p_fifo)) { - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "Servicing p_madw = %p\n", p_madw); - + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, + "Servicing p_madw = %p\n", p_madw); if (osm_log_is_active(p_vl->p_log, OSM_LOG_FRAMES)) osm_dump_dr_smp(p_vl->p_log, osm_madw_get_smp_ptr(p_madw), @@ -304,10 +301,8 @@ void osm_vl15_poll(IN osm_vl15_t * const p_vl) */ if (p_vl->p_stats->qp0_mads_outstanding_on_wire < (int32_t) p_vl->max_wire_smps) { - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "Signalling poller thread\n"); - + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, + "Signalling poller thread\n"); cl_event_signal(&p_vl->signal); } @@ -322,9 +317,7 @@ void osm_vl15_post(IN osm_vl15_t * const p_vl, IN osm_madw_t * const p_madw) CL_ASSERT(p_vl->state == OSM_VL15_STATE_READY); - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "Posting p_madw = 0x%p\n", p_madw); + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, "Posting p_madw = 0x%p\n", p_madw); /* Determine in which fifo to place the pending madw. @@ -337,11 +330,10 @@ void osm_vl15_post(IN osm_vl15_t * const p_vl, IN osm_madw_t * const p_madw) cl_qlist_insert_tail(&p_vl->ufifo, &p_madw->list_item); cl_spinlock_release(&p_vl->lock); - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "%u QP0 MADs on wire, %u QP0 MADs outstanding\n", - p_vl->p_stats->qp0_mads_outstanding_on_wire, - p_vl->p_stats->qp0_mads_outstanding); + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, + "%u QP0 MADs on wire, %u QP0 MADs outstanding\n", + p_vl->p_stats->qp0_mads_outstanding_on_wire, + p_vl->p_stats->qp0_mads_outstanding); osm_vl15_poll(p_vl); @@ -367,9 +359,8 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, /* first we handle the list of response MADs */ p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->ufifo); while (p_madw != (osm_madw_t *) cl_qlist_end(&p_vl->ufifo)) { - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "Releasing Response p_madw = %p\n", p_madw); + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, + "Releasing Response p_madw = %p\n", p_madw); osm_mad_pool_put(p_mad_pool, p_madw); @@ -379,9 +370,8 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, /* Request MADs we send out */ p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->rfifo); while (p_madw != (osm_madw_t *) cl_qlist_end(&p_vl->rfifo)) { - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, - "Releasing Request p_madw = %p\n", p_madw); + OSM_LOG(p_vl->p_log, OSM_LOG_DEBUG, + "Releasing Request p_madw = %p\n", p_madw); osm_mad_pool_put(p_mad_pool, p_madw); cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); diff --git a/opensm/opensm/osm_vl_arb_rcv.c b/opensm/opensm/osm_vl_arb_rcv.c index 05d54e2..725cc3b 100644 --- a/opensm/opensm/osm_vl_arb_rcv.c +++ b/opensm/opensm/osm_vl_arb_rcv.c @@ -120,13 +120,11 @@ void osm_vla_rcv_process(IN void *context, IN void *data) We do not mind if this is a result of a set or get - all we want is to update the subnet. */ - if (osm_log_is_active(sm->p_log, OSM_LOG_VERBOSE)) { - OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, - "Got GetResp(VLArb) block:%u port_num %u with GUID 0x%" - PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" - PRIx64 "\n", block_num, port_num, cl_ntoh64(port_guid), - cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); - } + OSM_LOG(sm->p_log, OSM_LOG_VERBOSE, + "Got GetResp(VLArb) block:%u port_num %u with GUID 0x%" + PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" + PRIx64 "\n", block_num, port_num, cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); /* Determine if we encountered a new Physical Port. -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Sat Jul 26 19:03:56 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 05:03:56 +0300 Subject: [ofa-general] [PATCH] opensm/osmtest: convert osm_log() to OSM_LOG() macro In-Reply-To: <20080727020306.GJ5067@sashak.voltaire.com> References: <20080709223030.GY12551@sashak.voltaire.com> <20080709223132.GZ12551@sashak.voltaire.com> <20080709223534.GA12551@sashak.voltaire.com> <20080727020306.GJ5067@sashak.voltaire.com> Message-ID: <20080727020356.GK5067@sashak.voltaire.com> Convert logging to use OSM_LOG() macro. Signed-off-by: Sasha Khapyorsky --- opensm/osmtest/osmt_inform.c | 179 ++--- opensm/osmtest/osmt_multicast.c | 1000 ++++++++--------------- opensm/osmtest/osmt_service.c | 300 +++----- opensm/osmtest/osmt_slvl_vl_arb.c | 60 +- opensm/osmtest/osmtest.c | 1640 ++++++++++++------------------------- 5 files changed, 1056 insertions(+), 2123 deletions(-) diff --git a/opensm/osmtest/osmt_inform.c b/opensm/osmtest/osmt_inform.c index 618b263..7257c7e 100644 --- a/opensm/osmtest/osmt_inform.c +++ b/opensm/osmtest/osmt_inform.c @@ -75,13 +75,11 @@ osmt_bind_inform_qp(IN osmtest_t * const p_osmt, OUT osmt_qp_ctx_t * p_qp_ctx) port_guid = p_osmt->local_port.port_guid; - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_bind_inform_qp: " - "Binding to port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Binding to port 0x%" PRIx64 "\n", + cl_ntoh64(port_guid)); /* obtain the hca name and port num from the guid */ - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_bind_inform_qp: " + OSM_LOG(p_log, OSM_LOG_DEBUG, "Finding CA and Port that owns port guid 0x%" PRIx64 "\n", port_guid); @@ -91,8 +89,7 @@ osmt_bind_inform_qp(IN osmtest_t * const p_osmt, OUT osmt_qp_ctx_t * p_qp_ctx) &hca_hndl, &hca_id[0], &hca_index, &port_num); if (mgt_ret != IB_MGT_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_bind_inform_qp: ERR 0109: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0109: " "Unable to obtain CA and port (%d).\n"); status = IB_ERROR; goto Exit; @@ -108,8 +105,7 @@ osmt_bind_inform_qp(IN osmtest_t * const p_osmt, OUT osmt_qp_ctx_t * p_qp_ctx) vapi_ret = osmt_mtl_init_opened_hca(&p_qp_ctx->qp_bind_hndl); if (vapi_ret != VAPI_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_bind_inform_qp: ERR 0114: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0114: " "Error initializing QP.\n"); status = IB_ERROR; goto Exit; @@ -129,20 +125,16 @@ osmt_bind_inform_qp(IN osmtest_t * const p_osmt, OUT osmt_qp_ctx_t * p_qp_ctx) memset(p_qp_ctx->p_send_buf, 0, MAD_BLOCK_SIZE); status = IB_SUCCESS; - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_bind_inform_qp: " - "Initialized QP:0x%X in VAPI Mode\n", + OSM_LOG(p_log, OSM_LOG_DEBUG, "Initialized QP:0x%X in VAPI Mode\n", p_qp_ctx->qp_bind_hndl.qp_id); - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_bind_inform_qp: " "Binding to IB_MGT SMI\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Binding to IB_MGT SMI\n"); /* we also need a QP0 handle for sending packets */ mgt_ret = IB_MGT_get_handle(hca_id, port_num, IB_MGT_SMI, &(p_qp_ctx->ib_mgt_qp0_handle)); if (IB_MGT_OK != mgt_ret) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_bind_inform_qp: ERR 0115: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0115: " "Error obtaining IB_MGT handle to SMI\n"); status = IB_ERROR; goto Exit; @@ -167,8 +159,7 @@ osmt_unbind_inform_qp(IN osmtest_t * const p_osmt, IN osmt_qp_ctx_t * p_qp_ctx) IB_MGT_release_handle(p_qp_ctx->ib_mgt_qp0_handle); - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_unbind_inform_qp: " "Unbind QP handles\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Unbind QP handles\n"); OSM_LOG_EXIT(&p_osmt->log); } @@ -208,14 +199,12 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, memcpy(p_ii, p_inform_info, sizeof(ib_inform_info_t)); if (reg_flag) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_reg_unreg_inform_info: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Subscribing InformInfo: Traps from lid:0x%X to 0x%X, trap num :0x%X\n", p_ii->lid_range_begin, p_ii->lid_range_end, p_ii->g_or_v.generic.trap_num); } else { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_reg_unreg_inform_info: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "UnSubscribing InformInfo: Traps from lid:0x%X to 0x%X\n", p_ii->lid_range_begin, p_ii->lid_range_end); } @@ -239,28 +228,24 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, /* --------------------- PREP ------------------------- */ if (osmt_mtl_mad_post_recv_bufs(&p_qp_ctx->qp_bind_hndl, p_qp_ctx->p_recv_buf, 1, /* but we need only one mad at a time */ GRH_LEN + MAD_BLOCK_SIZE, wrid) != 1) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0120: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0120: " "Error posting recv bufs\n"); status = IB_ERROR; goto Exit; } - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_reg_unreg_inform_info: " "Posted recv bufs\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Posted recv bufs\n"); vapi_ret = osmt_mtl_create_av(&p_qp_ctx->qp_bind_hndl, p_osmt->local_port.sm_lid, &avh); if (vapi_ret != VAPI_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0121: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0121: " "Error Preparing AVH (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; goto Exit; } - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_reg_unreg_inform_info: " "Prepared AVH\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Prepared AVH\n"); if (osm_log_is_active(p_log, OSM_LOG_DEBUG)) { osm_dump_sa_mad(p_log, (ib_sa_mad_t *) (p_qp_ctx->p_send_buf), @@ -282,8 +267,7 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, OSMT_MTL_REVERSE_QP1_WELL_KNOWN_Q_KEY, avh); if (vapi_ret != VAPI_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0122: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0122: " "Error sending mad (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; @@ -294,8 +278,7 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, p_qp_ctx->qp_bind_hndl.sq_cq_hndl, &wc_desc, 20, 10000, NULL); if (vapi_ret != VAPI_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0123: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0123: " "Error getting send completion (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; @@ -303,15 +286,13 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, } if (wc_desc.status != VAPI_SUCCESS) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0124: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0124: " "Error on send completion (%s) (%d)\n", VAPI_strerror_sym(wc_desc.status), wc_desc.status); status = IB_ERROR; goto Exit; } - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_reg_unreg_inform_info: " "Sent MAD\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Sent MAD\n"); /* --------------------- RECV ------------------------- */ vapi_ret = osmt_mtl_mad_poll4cqe(p_qp_ctx->qp_bind_hndl.hca_hndl, @@ -321,8 +302,7 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, if (vapi_ret == VAPI_CQ_EMPTY) { status = IB_TIMEOUT; } else { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0125: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0125: " "Error receiving mad (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; @@ -334,17 +314,14 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, p_sa_mad = (ib_sa_mad_t *) (p_qp_ctx->p_recv_buf + GRH_LEN); if (p_sa_mad->status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: " - "Remote error = %s\n", + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str((ib_mad_t *) p_sa_mad)); status = IB_REMOTE_ERROR; goto Exit; } if (p_sa_mad->method != IB_MAD_METHOD_GET_RESP) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Expected IB_MAD_METHOD_GET_RESP but got:(%X)\n", p_sa_mad->method); status = IB_REMOTE_ERROR; @@ -352,8 +329,7 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, } if (p_sa_mad->attr_id != IB_MAD_ATTR_INFORM_INFO) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Expected IB_MAD_ATTR_INFORM_INFO but got:(%X)\n", cl_ntoh16(p_sa_mad->attr_id)); status = IB_REMOTE_ERROR; @@ -362,8 +338,7 @@ osmt_reg_unreg_inform_info(IN osmtest_t * p_osmt, p_ii = ib_sa_mad_get_payload_ptr(p_sa_mad); if (!p_ii->subscribe) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_reg_unreg_inform_info: ERR 0126: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0126: " "Subscribe/Unsubscribe Failed\n"); status = IB_REMOTE_ERROR; } @@ -397,8 +372,7 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(p_log); - osm_log(p_log, OSM_LOG_INFO, - "osmt_send_trap_wait_for_forward: " + OSM_LOG(p_log, OSM_LOG_INFO, "Sending Traps to QP0 of SA LID:0x%X\n", p_osmt->local_port.sm_lid); @@ -421,14 +395,12 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, /* --------------------- PREP ------------------------- */ if (osmt_mtl_mad_post_recv_bufs(&p_qp_ctx->qp_bind_hndl, p_qp_ctx->p_recv_buf, 1, /* we need to receive both trap repress and report */ GRH_LEN + MAD_BLOCK_SIZE, wrid) != 1) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 0127: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0127: " "Error posting recv bufs\n"); status = IB_ERROR; goto Exit; } - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_send_trap_wait_for_forward: " "Posted recv bufs\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Posted recv bufs\n"); av.dlid = p_osmt->local_port.sm_lid; av.grh_flag = FALSE; @@ -438,14 +410,9 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, av.src_path_bits = 1; /* p_mad_addr->path_bits; */ av.sl = 0; /* p_mad_addr->addr_type.gsi.service_level; */ - if (osm_log_is_active(p_log, OSM_LOG_DEBUG)) { - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_send_trap_wait_for_forward: " - "av.dlid 0x%X, " - "av.static_rate %d, " - "av.path_bits %d\n", - cl_ntoh16(av.dlid), av.static_rate, av.src_path_bits); - } + OSM_LOG(p_log, OSM_LOG_DEBUG, + "av.dlid 0x%X, av.static_rate %d, av.path_bits %d\n", + cl_ntoh16(av.dlid), av.static_rate, av.src_path_bits); /* send it */ mgt_res = IB_MGT_send_mad(p_qp_ctx->ib_mgt_qp0_handle, p_smp, /* actual payload */ @@ -453,8 +420,7 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, wrid, /* casting the mad wrapper pointer for err cb */ p_osmt->opt.transaction_timeout); if (mgt_res != IB_MGT_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 0128: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0128: " "Error sending mad (%d)\n", mgt_res); status = IB_ERROR; goto Exit; @@ -464,18 +430,15 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, osmt_mtl_create_av(&p_qp_ctx->qp_bind_hndl, p_osmt->local_port.sm_lid, &avh); if (vapi_ret != VAPI_OK) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 0129: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0129: " "Error Preparing AVH (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; goto Exit; } - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_send_trap_wait_for_forward: " "Prepared AVH\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Prepared AVH\n"); - osm_log(p_log, OSM_LOG_DEBUG, - "osmt_send_trap_wait_for_forward: " "Trap MAD Sent\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Trap MAD Sent\n"); /* --------------------- RECV ------------------------- */ vapi_ret = osmt_mtl_mad_poll4cqe(p_qp_ctx->qp_bind_hndl.hca_hndl, @@ -483,14 +446,12 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, &wc_desc, 200, 10000, &avh); if (vapi_ret != VAPI_SUCCESS) { if (vapi_ret == VAPI_CQ_EMPTY) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 0130: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0130: " "Timeout receiving mad (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_TIMEOUT; } else { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 0131: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0131: " "Error receiving mad (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; @@ -503,21 +464,16 @@ osmt_send_trap_wait_for_forward(IN osmtest_t * const p_osmt, if (p_sa_mad->method == IB_MAD_METHOD_REPORT) { if (p_sa_mad->attr_id == IB_MAD_ATTR_NOTICE) { - osm_log(p_log, OSM_LOG_INFO, - "osmt_send_trap_wait_for_forward: " - "Received the Report!\n"); + OSM_LOG(p_log, OSM_LOG_INFO, "Received the Report!\n"); status = IB_SUCCESS; } else { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 1020" + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 1020" "Did not receive a Report(Notice) but attr:%d\n", - cl_ntoh16(p_sa_mad->attr_id) - ); + cl_ntoh16(p_sa_mad->attr_id)); status = IB_ERROR; } } else { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_send_trap_wait_for_forward: ERR 1020" + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 1020" "Received an Unexpected Method:%d\n", p_smp->method); status = IB_ERROR; } @@ -543,8 +499,7 @@ osmt_trap_wait(IN osmtest_t * const p_osmt, IN osmt_qp_ctx_t * p_qp_ctx) OSM_LOG_ENTER(p_log); - osm_log(p_log, OSM_LOG_INFO, - "osmt_trap_wait: " + OSM_LOG(p_log, OSM_LOG_INFO, "Waiting for Traps under QP:0x%X of SA LID:0x%X\n", cl_ntoh16(p_osmt->local_port.sm_lid)); @@ -557,14 +512,12 @@ osmt_trap_wait(IN osmtest_t * const p_osmt, IN osmt_qp_ctx_t * p_qp_ctx) 10000, NULL); if (vapi_ret != VAPI_SUCCESS) { if (vapi_ret == VAPI_CQ_EMPTY) { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_trap_wait: ERR 0130: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0130: " "Timeout receiving mad (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_TIMEOUT; } else { - osm_log(p_log, OSM_LOG_ERROR, - "osmt_trap_wait: ERR 0131: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0131: " "Error receiving mad (%s)\n", VAPI_strerror_sym(vapi_ret)); status = IB_ERROR; @@ -577,20 +530,17 @@ osmt_trap_wait(IN osmtest_t * const p_osmt, IN osmt_qp_ctx_t * p_qp_ctx) if (p_sa_mad->method == IB_MAD_METHOD_REPORT) { if (p_sa_mad->attr_id == IB_MAD_ATTR_NOTICE) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_trap_wait: " "Received the Report!\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "Received the Report!\n"); status = IB_SUCCESS; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_trap_wait: ERR 1020" + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 1020" "Did not receive a Report(Notice) but attr:%d\n", - cl_ntoh16(p_sa_mad->attr_id) - ); + cl_ntoh16(p_sa_mad->attr_id)); status = IB_ERROR; } } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_trap_wait: ERR 1020" + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 1020" "Received an Unexpected Method:%d\n", p_smp->method); status = IB_ERROR; } @@ -671,14 +621,12 @@ ib_api_status_t osmt_run_inform_info_flow(IN osmtest_t * const p_osmt) /* WAS IB_REMOTE_ERROR */ if (status != IB_REMOTE_ERROR) { if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_inform_info_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Error during UnSubscribe: (%s)\n", ib_get_err_str(status)); goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_inform_info_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Expected Failure to UnSubscribe non existing InformInfo\n"); status = IB_ERROR; goto Exit; @@ -694,8 +642,7 @@ ib_api_status_t osmt_run_inform_info_flow(IN osmtest_t * const p_osmt) /* send a trap through QP0 and wait on QPN */ status = osmt_send_trap_wait_for_forward(p_osmt, &qp_ctx); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_inform_info_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Error during Send Trap and Wait For Report: (%s)\n", ib_get_err_str(status)); goto Exit; @@ -705,15 +652,13 @@ ib_api_status_t osmt_run_inform_info_flow(IN osmtest_t * const p_osmt) status = osmt_reg_unreg_inform_info(p_osmt, &qp_ctx, &inform_info, 0); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_inform_info_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Error during UnSubscribe: (%s)\n", ib_get_err_str(status)); goto Exit; } else { if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_inform_info_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Remote Error during UnSubscribe\n"); status = IB_ERROR; goto Exit; @@ -762,15 +707,13 @@ ib_api_status_t osmt_run_trap64_65_flow(IN osmtest_t * const p_osmt) /*--------------------- PREP -------------------------*/ if (osmt_mtl_mad_post_recv_bufs(&qp_ctx.qp_bind_hndl, qp_ctx.p_recv_buf, 1, /* we need to receive the report */ GRH_LEN + MAD_BLOCK_SIZE, 1) != 1) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_trap64_65_flow: ERR 0127: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0127: " "Error posting recv bufs for trap 64\n"); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmt_run_trap64_65_flow: " "Posted recv bufs for trap 64\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "Posted recv bufs for trap 64\n"); /* init the inform info */ osmt_init_inform_info_by_trap(p_osmt, cl_hton16(65), &inform_info); @@ -784,14 +727,12 @@ ib_api_status_t osmt_run_trap64_65_flow(IN osmtest_t * const p_osmt) /*--------------------- PREP -------------------------*/ if (osmt_mtl_mad_post_recv_bufs(&qp_ctx.qp_bind_hndl, qp_ctx.p_recv_buf, 1, /* we need to reveive the report */ GRH_LEN + MAD_BLOCK_SIZE, 1) != 1) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_trap64_65_flow: ERR 0127: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0127: " "Error posting recv bufs for trap 65\n"); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmt_run_trap64_65_flow: " "Posted recv bufs for trap 65\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "Posted recv bufs for trap 65\n"); /* Sleep for x seconds in order to allow external script trap generation */ #if 0 @@ -801,8 +742,7 @@ ib_api_status_t osmt_run_trap64_65_flow(IN osmtest_t * const p_osmt) /* wait for a trap on QPN */ status = osmt_trap_wait(p_osmt, &qp_ctx); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_trap64_65_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Error during Send Trap and Wait For Report: (%s)\n", ib_get_err_str(status)); goto Exit; @@ -812,8 +752,7 @@ ib_api_status_t osmt_run_trap64_65_flow(IN osmtest_t * const p_osmt) status = osmt_reg_unreg_inform_info(p_osmt, &qp_ctx, &inform_info, 0); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_trap64_65_flow:" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Error during UnSubscribe: (%s)\n", ib_get_err_str(status)); goto Exit; diff --git a/opensm/osmtest/osmt_multicast.c b/opensm/osmtest/osmt_multicast.c index a4f80b9..6725d8f 100644 --- a/opensm/osmtest/osmt_multicast.c +++ b/opensm/osmtest/osmt_multicast.c @@ -83,8 +83,7 @@ static void __osmt_print_all_multicast_records(IN osmtest_t * const p_osmt) status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS || context.result.status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "__osmt_print_all_multicast_records: ERR 02B5: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B5: " "Failed getting the multicast groups records - %s/%s\n", ib_get_err_str(status), ib_get_err_str(context.result.status)); @@ -107,8 +106,7 @@ static void __osmt_print_all_multicast_records(IN osmtest_t * const p_osmt) status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS || context.result.status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "__osmt_print_all_multicast_records: ERR 02B6: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B6: " "Failed getting the multicast group members records - %s/%s\n", ib_get_err_str(status), ib_get_err_str(context.result.status)); @@ -191,8 +189,7 @@ ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_mcast: ERR 0203: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0203: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -200,12 +197,10 @@ ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_mcast: ERR 0264: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0264: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_mcast: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s.\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -229,8 +224,8 @@ ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) cl_list_init(&mgids_list, num_recs); p_mgids_list = &mgids_list; num_recs = context.result.result_cnt; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_query_mcast: " "Received %u records\n", num_recs); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", + num_recs); for (i = 0; i < num_recs; i++) { p_rec = osmv_get_query_result(context.result.p_result_madw, i); @@ -240,11 +235,10 @@ ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) /* If returns iterator other than end of list, same mgid exists already */ if (p_mgids_res != cl_list_end(p_mgids_list)) { char gid_str[INET6_ADDRSTRLEN]; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_mcast: ERR 0265: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0265: " "MCG MGIDs are the same - invalid MGID : %s\n", inet_ntop(AF_INET6, p_rec->mgid.raw, gid_str, - sizeof gid_str)); + sizeof gid_str)); status = IB_ERROR; goto Exit; @@ -252,16 +246,14 @@ ib_api_status_t osmt_query_mcast(IN osmtest_t * const p_osmt) osm_dump_mc_record(&p_osmt->log, p_rec, OSM_LOG_VERBOSE); cl_status = cl_list_insert_head(p_mgids_list, &(p_rec->mgid)); if (cl_status) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_mcast: ERR 0205: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0205: " "Could not add MGID to cl_list\n"); status = IB_ERROR; goto Exit; } p_mgrp = (osmtest_mgrp_t *) malloc(sizeof(*p_mgrp)); if (!p_mgrp) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_mcast: ERR 0204: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0204: " "Could not allocate new MCG\n"); status = IB_ERROR; goto Exit; @@ -320,16 +312,15 @@ osmt_send_mcast_request(IN osmtest_t * const p_osmt, } else if (is_set == 0) { req.query_type = OSMV_QUERY_UD_MULTICAST_DELETE; } else if (is_set == 0xee) { - - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_send_mcast_request: Set USER DEFINED QUERY\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Set USER DEFINED QUERY\n"); req.query_type = OSMV_QUERY_USER_DEFINED; user.method = IB_MAD_METHOD_GET; user.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; user.attr_offset = ib_get_attr_offset(sizeof(ib_member_rec_t)); } else if (is_set == 0xff) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_send_mcast_request: Set USER DEFINED QUERY\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Set USER DEFINED QUERY\n"); req.query_type = OSMV_QUERY_USER_DEFINED; user.method = IB_MAD_METHOD_SET; user.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; @@ -356,8 +347,7 @@ osmt_send_mcast_request(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_send_mcast_request: ERR 0206: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0206: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -370,12 +360,10 @@ osmt_send_mcast_request(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_send_mcast_request: ERR 0224: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0224: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_send_mcast_request: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -550,12 +538,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) OSM_LOG_ENTER(&p_osmt->log); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " "GetTable of all current MCGs...\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "GetTable of all current MCGs...\n"); status = osmt_query_mcast(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 2FF " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 2FF " "GetTable of all records has failed!\n"); goto Exit; } @@ -572,8 +558,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* Only when we are on single mode check flow - do the count comparison, otherwise skip */ if (p_osmt->opt.mmode == 1 || p_osmt->opt.mmode == 3) { start_cnt = cl_qmap_count(p_mgrp_mlid_tbl); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow (start): " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "(start): " "Number of MC Records found in SA DB is %d\n", start_cnt); } @@ -596,26 +581,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (IS_IPOIB_MGID(&p_mgrp->mcmember_rec.mgid)) { start_ipoib_cnt++; } else { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Non-IPoIB MC Groups exist: mgid=%s\n", inet_ntop(AF_INET6, - p_mgrp->mcmember_rec.mgid.raw, - gid_str, sizeof gid_str)); + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str)); mcg_outside_test_cnt++; } p_mgrp = (osmtest_mgrp_t *) cl_qmap_next(&p_mgrp->map_item); } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Found %d non-IPoIB MC Groups\n", mcg_outside_test_cnt); if (start_ipoib_cnt) { /* o15-0.2.4 - Check a join request to already created MCG */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Found IPoIB MC Group, so we run SilverStorm Bug Flow...\n"); /* Try to join first like IPoIB of SilverStorm */ memcpy(&mc_req_rec.mgid, &osm_ipoib_good_mgid, @@ -631,23 +613,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Joining an existing IPoIB multicast group\n"); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " - "Sent Join request with :\n\t\tport_gid=%s" - ", mgid=%s\n" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "Sent Join request with :\n\t\tport_gid=%s, mgid=%s\n" "\t\tjoin state= 0x%x, response is : %s\n", inet_ntop(AF_INET6, mc_req_rec.port_gid.raw, - gid_str, sizeof gid_str), + gid_str, sizeof gid_str), inet_ntop(AF_INET6, mc_req_rec.mgid.raw, - gid_str2, sizeof gid_str2), + gid_str2, sizeof gid_str2), (mc_req_rec.scope_state & 0x0F), ib_get_err_str(status)); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02B3: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B3: " "Failed joining existing IPoIB MCGroup - got %s\n", ib_get_err_str(status)); goto Exit; @@ -663,8 +641,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) has included some more fields but we know that problem was genereated by the RATE */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Received attributes of MCG : \n\t\tMTU=0x%02X, RATE=0x%02X\n", p_mc_res->mtu, p_mc_res->rate); @@ -686,25 +663,21 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_MTU | IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Sending attributes of MCG : \n\t\tMTU=0x%02X, RATE=0x%02X\n", mc_req_rec.mtu, mc_req_rec.rate); status = osmt_send_mcast_request(p_osmt, 0xff, /* User Defined query */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Sent Join request using response values, response is : %s\n", ib_get_err_str(status)); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02EF: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02EF: " "Query as Full Member of already existing " "ipoib group gid %s has failed\n", inet_ntop(AF_INET6, mc_req_rec.mgid.raw, - gid_str, sizeof gid_str)); - + gid_str, sizeof gid_str)); goto Exit; } /* We do not want to leave the MCG since its IPoIB */ @@ -713,23 +686,20 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /**************************************************************************/ /* Check Get with invalid mlid */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " "Checking Get with invalid mlid...\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "Checking Get with invalid mlid...\n"); /* Request Get */ ib_member_set_join_state(&mc_req_rec, IB_MC_REC_STATE_FULL_MEMBER); mc_req_rec.mlid = invalid_mlid; comp_mask = IB_MCR_COMPMASK_MLID; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 0xee, /* User Defined query Get */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow : ERR 2E0 " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 2E0 " "SubnAdmGet with invalid mlid 0x%x succeeded\n", cl_ntoh16(mc_req_rec.mlid)); status = IB_ERROR; @@ -741,8 +711,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /**************************************************************************/ /* Check Get with invalid port guid */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Get with invalid port guid (0x0) but valid interface ID : 0x%" PRIx64 "...\n", cl_ntoh64(mc_req_rec.port_gid.unicast.interface_id)); @@ -753,16 +722,13 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) sizeof(ib_net64_t)); comp_mask = IB_MCR_COMPMASK_GID; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 0xee, /* User Defined query Get */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow : ERR 2E4 " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 2E4 " "SubnAdmGet with invalid port guid succeeded\n"); status = IB_ERROR; goto Exit; @@ -775,8 +741,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* o15.0.1.3: */ /* - Request Join with insufficient comp_mask */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with insufficient comp mask qkey & pkey (o15.0.1.3)...\n"); /* no MGID */ @@ -790,28 +755,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02EE: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02EE: " "Expectedd REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with insufficient comp mask - sl (15.0.1.3)...\n"); /* no MGID */ @@ -827,22 +787,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02ED: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02ED: " "Expectedd REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -855,8 +811,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_mc_res = ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with insufficient comp mask - flow label (o15.0.1.3)...\n"); /* Request Join */ @@ -870,22 +825,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02EC: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02EC: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -894,8 +845,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_mc_res = ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with insufficient comp mask - tclass (o15.0.1.3)...\n"); /* Request Join */ @@ -911,22 +861,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* IB_MCR_COMPMASK_TCLASS | Intentionally missed to raise an error */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02EA: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02EA: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -935,8 +881,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_mc_res = ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with insufficient comp mask - tclass qkey (o15.0.1.3)...\n"); /* no MGID */ @@ -952,30 +897,25 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* IB_MCR_COMPMASK_TCLASS | intentionaly missed to raise the error */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02E9: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02E9: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* o15.0.1.8: */ /* - Request join with irrelevant RATE : get a ERR_INSUFFICIENT_COMPONENTS */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with unrealistic rate (o15.0.1.8)...\n"); /* impossible requested rate */ @@ -985,28 +925,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0207: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0207: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Check Valid value which is unreasonable now */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with unrealistic rate 120GB (o15.0.1.8)...\n"); /* impossible requested rate */ @@ -1016,28 +951,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0208: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0208: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Check Valid value which is unreasonable now */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with less than min rate 2.5GB (o15.0.1.8)...\n"); /* impossible requested rate */ @@ -1047,28 +977,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02AB: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02AB: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Checking above max value of MTU which is impossible */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with unrealistic mtu : \n\t\tmore than 4096 -" " max (o15.0.1.8)...\n"); @@ -1078,17 +1003,14 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_MTU_SEL | IB_MCR_COMPMASK_MTU; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02AC: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02AC: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) @@ -1098,8 +1020,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) } /* Checking below min value of MTU which is impossible */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with unrealistic mtu : \n\t\tless than 256 -" " min (o15.0.1.8)...\n"); @@ -1109,27 +1030,22 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_MTU_SEL | IB_MCR_COMPMASK_MTU; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02AD: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02AD: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with unrealistic mtu (o15.0.1.8)...\n"); /* impossible requested mtu */ @@ -1138,29 +1054,24 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_MTU_SEL | IB_MCR_COMPMASK_MTU; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02AE: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02AE: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } #if 0 /* Currently PacketLifeTime isn't checked in opensm */ /* Check PacketLifeTime as 0 */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create with unrealistic packet life value less than 0 (o15.0.1.8)...\n"); /* impossible requested packet life */ @@ -1169,21 +1080,17 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask = IB_MCR_COMPMASK_GID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_QKEY | IB_MCR_COMPMASK_PKEY | IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_LIFE | IB_MCR_COMPMASK_LIFE_SEL; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02AF: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02AF: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_REQ_INVALID got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -1193,8 +1100,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* - Create an MGID by asking for a join with MGID = 0 */ /* providing P_Key, Q_Key, SL, FlowLabel, Tclass. */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given MGID=0 skip service level (o15.0.1.4)...\n"); osmt_init_mc_query_rec(p_osmt, &mc_req_rec); @@ -1214,22 +1120,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02A8: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02A8: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -1238,33 +1140,28 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_query_mcast(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02AA: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02AA: " "Could not get all MC Records in subnet, got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Only when we are on single mode check flow - do the count comparison, otherwise skip */ if (p_osmt->opt.mmode == 1 || p_osmt->opt.mmode == 3) { middle_cnt = cl_qmap_count(&p_osmt->exp_subn.mgrp_mlid_tbl); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow (post false create): " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "(post false create): " "Number of MC Records found in SA DB is %d\n", middle_cnt); if (middle_cnt != start_cnt) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Got different number of records stored in SA DB (before any creation)\n" "Instead of %d got %d\n", start_cnt, middle_cnt); } } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given MGID=0 skip Qkey and Pkey (o15.0.1.4)...\n"); osmt_init_mc_query_rec(p_osmt, &mc_req_rec); @@ -1282,22 +1179,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_SL | IB_MCR_COMPMASK_FLOW | IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_TCLASS | /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02A7: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02A7: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -1306,8 +1199,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_query_mcast(p_osmt); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given MGID=0 skip TClass (o15.0.1.4)...\n"); osmt_init_mc_query_rec(p_osmt, &mc_req_rec); @@ -1330,28 +1222,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* all above are required */ IB_MCR_COMPMASK_RATE_SEL | IB_MCR_COMPMASK_RATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR || ((ib_net16_t) (res_sa_mad.status & IB_SMP_STATUS_MASK)) != IB_SA_MAD_STATUS_INSUF_COMPS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02A6: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02A6: " "Expected REMOTE ERROR IB_SA_MAD_STATUS_INSUF_COMPS got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given MGID=0 valid Set several options :\n\t\t" "First above min RATE, Second less than max RATE\n\t\t" "Third above min MTU, Second less than max MTU\n\t\t" @@ -1369,23 +1256,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02A5: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02A5: " "Failed to create MCG for MGID=0 with higher than minimum RATE - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1400,23 +1283,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0211: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0211: " "Failed to create MCG for MGID=0 with less than highest RATE - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1430,23 +1309,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0238: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0238: " "Failed to create MCG for MGID=0 with less than highest MTU - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1459,31 +1334,26 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0239: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0239: " "Failed to create MCG for MGID=0 with higher than lowest MTU - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); /* Good Flow - mgid is 0 while giving all required fields for join : P_Key, Q_Key, SL, FlowLabel, Tclass */ /* Using Exact feasible MTU & RATE */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Using Exact feasible MTU & RATE: " "MTU = 0x%02X, RATE = 0x%02X\n", mtu_phys, rate_phys); @@ -1499,31 +1369,26 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0240: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0240: " "Failed to create MCG for MGID=0 with exact MTU & RATE - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); /* Good Flow - mgid is 0 while giving all required fields for join : P_Key, Q_Key, SL, FlowLabel, Tclass */ /* Using Exact feasible RATE */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Using Exact feasible RATE: 0x%02X\n", rate_phys); mc_req_rec.rate = rate_phys; @@ -1534,31 +1399,26 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0241: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0241: " "Failed to create MCG for MGID=0 with exact RATE - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); /* Good Flow - mgid is 0 while giving all required fields for join : P_Key, Q_Key, SL, FlowLabel, Tclass */ /* Using Exact feasible MTU */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Using Exact feasible MTU: 0x%02X\n", mtu_phys); mc_req_rec.mtu = mtu_phys; @@ -1569,30 +1429,25 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0242: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0242: " "Failed to create MCG for MGID=0 with exact MTU - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); /* o15.0.1.5: */ /* - Check the returned MGID is valid. (p 804) */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating resulting MGID (o15.0.1.5)...\n"); /* prefix 0xFF1 Scope 0xA01B */ /* Since we did not directly specified SCOPE in comp mask @@ -1601,20 +1456,17 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) (p_mc_res->mgid.multicast.header[1] != 0x12) || (p_mc_res->mgid.multicast.raw_group_id[0] != 0xA0) || (p_mc_res->mgid.multicast.raw_group_id[1] != 0x1B)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0209: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0209: " "Validating MGID failed. MGID:%s\n", inet_ntop(AF_INET6, p_mc_res->mgid.raw, gid_str, - sizeof gid_str) - ); + sizeof gid_str)); status = IB_ERROR; goto Exit; } /* Good Flow - mgid is 0 while giving all required fields for join : P_Key, Q_Key, SL, FlowLabel, Tclass */ /* Using feasible GREATER_THAN 0 packet lifitime */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given MGID=0 (o15.0.1.4)...\n"); status = osmt_query_mcast(p_osmt); @@ -1636,23 +1488,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0210: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0210: " "Failed to create MCG for MGID=0 - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1661,34 +1509,28 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) osmt_init_mc_query_rec(p_osmt, &mc_req_rec); mc_req_rec.mgid = good_mgid; - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given valid MGID=%s (o15.0.1.6)...\n", inet_ntop(AF_INET6, mc_req_rec.mgid.raw, gid_str, - sizeof gid_str)); + sizeof gid_str)); /* Before creation, need to check that this group doesn't exist */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Verifying that MCGroup with this MGID doesn't exist by trying to Join it (o15.0.1.13)...\n"); ib_member_set_join_state(&mc_req_rec, IB_MC_REC_STATE_NON_MEMBER); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, /* join */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0301: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0301: " "Tried joining group that shouldn't have existed - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -1696,21 +1538,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* Set State to full member to allow group creation */ ib_member_set_join_state(&mc_req_rec, IB_MC_REC_STATE_FULL_MEMBER); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Now creating group with given valid MGID=%s (o15.0.1.6)...\n", inet_ntop(AF_INET6, mc_req_rec.mgid.raw, gid_str, - sizeof gid_str)); + sizeof gid_str)); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0211: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0211: " "Failed to create MCG for MGID=%s (o15.0.1.6) - got %s/%s\n", inet_ntop(AF_INET6, good_mgid.raw, gid_str, - sizeof gid_str), - ib_get_err_str(status), + sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } @@ -1718,122 +1557,101 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating resulting MGID (o15.0.1.6)...\n"); /* prefix 0xFF1 Scope 0xA01B */ if ((p_mc_res->mgid.multicast.header[0] != 0xFF) || (p_mc_res->mgid.multicast.header[1] != 0x12) || /* HACK hardcoded scope = 0x02 */ (p_mc_res->mgid.multicast.raw_group_id[0] != 0xA0) || (p_mc_res->mgid.multicast.raw_group_id[1] != 0x1C)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0212: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0212: " "Validating MGID failed. MGID:%s\n", inet_ntop(AF_INET6, p_mc_res->mgid.raw, gid_str, - sizeof gid_str) - ); + sizeof gid_str)); status = IB_ERROR; goto Exit; } /* - Try to create a new MCG with invalid MGID : get back ERR_REQ_INVALID */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD MGID=0xFA..... (o15.0.1.6)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid.raw[0] = 0xFA; status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0213: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0213: " "Failed to recognize MGID error for MGID=0xFA - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* - Try again with MGID prefix = 0xA01B (maybe 0x1BA0 little or big ?) */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD MGID=0xFF12A01B..... with link-local scope (o15.0.1.6)...\n"); mc_req_rec.mgid.raw[0] = 0xFF; mc_req_rec.mgid.raw[3] = 0x1B; comp_mask = comp_mask | IB_MCR_COMPMASK_SCOPE; mc_req_rec.scope_state = mc_req_rec.scope_state & 0x2F; /* local scope */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0214: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0214: " "Failed to recognize MGID error for A01B with link-local bit (status %s) (rem status %s)\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Change the mgid prefix - get back ERR_REQ_INVALID */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD MGID PREFIX=0xEF... (o15.0.1.6)...\n"); mc_req_rec.mgid = good_mgid; mc_req_rec.mgid.raw[0] = 0xEF; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0215: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0215: " "Failed to recognize MGID PREFIX error for MGID=0xEF - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Change the scope to reserved - get back VALID REQ */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking local scope with full member \n\t\tand valid mgid %s" " ... (o15.0.1.6)...\n", inet_ntop(AF_INET6, mc_req_rec.mgid.raw, gid_str, - sizeof gid_str)); + sizeof gid_str)); mc_req_rec.mgid = good_mgid; @@ -1842,12 +1660,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0216: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0216: " "Failed to create MCG for MGID=%s - got %s/%s\n", inet_ntop(AF_INET6, good_mgid.raw, gid_str, - sizeof gid_str), - ib_get_err_str(status), + sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } @@ -1855,22 +1671,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); /* Change the flags to invalid value 0x2 - get back INVALID REQ */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking invalid flags=0xFF 22 ... (o15.0.1.6)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = good_mgid; @@ -1879,25 +1691,21 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0217: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0217: " "Failed to recognize create with invalid flags value 0x2 - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Change the MGID to link local MGID - get back VALID REQ */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking link local MGID 0xFF02:0:0:0:0:0:0:1 (o15.0.1.6)...\n"); mc_req_rec.mgid = osm_link_local_mgid; @@ -1905,23 +1713,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0218: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0218: " "Failed to create MCG for MGID=0xFF02:0:0:0:0:0:0:1 - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -1931,12 +1735,10 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* o15.0.1.9 */ /* - Create MCG with Invalid JoinState.FullMember != 1 : get ERR_REQ_INVALID */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking new MGID with invalid join state (o15.0.1.9)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = good_mgid; mc_req_rec.mgid.raw[12] = 0xFF; @@ -1944,24 +1746,20 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0219: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0219: " "Failed to recognize create with JoinState != FullMember - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Lets try a valid join scope state */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking new MGID with valid join state (o15.0.1.9)...\n"); mc_req_rec.mgid = good_mgid; @@ -1971,33 +1769,27 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0220: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0220: " "Failed to create MCG with valid join state 0x3 - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); /* Lets try another invalid join scope state */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking new MGID with invalid join state (o15.0.1.9)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); /* We have created a new MCG so now we need different mgid when cresting group otherwise it will be counted as join request . */ mc_req_rec.mgid = good_mgid; @@ -2007,24 +1799,20 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0221: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0221: " "Failed to recognize create with JoinState != FullMember - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Lets try another valid join scope state */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking new MGID creation with valid join state (o15.0.1.9)...\n"); mc_req_rec.mgid = good_mgid; @@ -2036,23 +1824,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0222: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0222: " "Failed to create MCG with valid join state 0xF - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* Save the mlid created in test_created_mlids map */ p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Created MGID:%s MLID:0x%04X\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, p_recvd_rec->mgid.raw, gid_str, - sizeof gid_str), - cl_ntoh16(p_recvd_rec->mlid)); + sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), p_recvd_rec); @@ -2060,8 +1844,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) checked by SilverStorm bug o15-0.2.4, never the less recheck */ /* o15-0.2.4 - Check a join request to already created MCG */ p_mc_res = ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " "Check o15-0.2.4 statement...\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Check o15-0.2.4 statement...\n"); /* Try to join */ memcpy(&mc_req_rec.mgid, &p_mc_res->mgid, sizeof(ib_gid_t)); /* Request Join */ @@ -2073,8 +1856,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 0x1, /* User Defined query */ &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02CC: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02CC: " "Failed to join MCG with valid req, returned status = %s\n", ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; @@ -2083,8 +1865,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* o15.0.1.11: */ /* - Try to join into a MGID that exists with JoinState=SendOnlyMember - */ /* see that it updates JoinState. What is the routing change? */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Retry of existing MGID - See JoinState update (o15.0.1.11)...\n"); mc_req_rec.mgid = good_mgid; @@ -2093,22 +1874,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02CD: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02CD: " "Failed to update existing MGID - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating Join State update with NonMember (o15.0.1.11)...\n"); if (p_mc_res->scope_state != 0x23) { /* scope is LSB */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02CE: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02CE: " "Validating JoinState update failed. Expected 0x23 got: 0x%02X\n", p_mc_res->scope_state); status = IB_ERROR; @@ -2116,8 +1893,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) } /* Try delete current join state then update it with another value */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking JoinState update request should return 0x22 (o15.0.1.11)...\n"); mc_req_rec.rate = @@ -2125,19 +1901,16 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) mc_req_rec.mgid = good_mgid; /* link-local scope, non member (so we should not be able to delete) */ /* but the FullMember bit should be gone */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Partially delete JoinState (o15.0.1.14)...\n"); mc_req_rec.scope_state = 0x22; status = osmt_send_mcast_request(p_osmt, 0, &mc_req_rec, comp_mask, &res_sa_mad); if ((status != IB_SUCCESS) || (p_mc_res->scope_state != 0x21)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02CF: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02CF: " "Failed to partially update JoinState : %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -2149,22 +1922,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C0: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C0: " "Failed to update existing MCG - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating Join State update with Send Only Member (o15.0.1.11)...\n"); if (p_mc_res->scope_state != 0x25) { /* scope is MSB */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C1: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C1: " "Validating JoinState update failed. Expected 0x25 got: 0x%02X\n", p_mc_res->scope_state); status = IB_ERROR; @@ -2176,23 +1945,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C2: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C2: " "Failed to update existing MGID - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating Join State update with Full Member\n\t\t" "to an existing 0x5 state MCG (o15.0.1.11)...\n"); if (p_mc_res->scope_state != 0x25) { /* scope is LSB */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C3: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C3: " "Validating JoinState update failed. Expected 0x25 got: 0x%02X\n", p_mc_res->scope_state); status = IB_ERROR; @@ -2205,29 +1970,25 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C4: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C4: " "Failed to update existing MGID - got %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating Join State update with Non Member\n\t\t" "to an existing 0x5 state MCG (o15.0.1.11)...\n"); if (p_mc_res->scope_state != 0x27) { /* scope is LSB */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C5: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C5: " "Validating JoinState update failed. Expected 0x27 got: 0x%02X\n", p_mc_res->scope_state); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "DEBUG - Current scope_state value : 0x%02X...\n", p_mc_res->scope_state); @@ -2241,11 +2002,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* - Try joining with rate that does not exist in any MCG - ERR_REQ_INVALID */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD RATE when connecting to existing MGID (o15.0.1.13)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = good_mgid; mc_req_rec.rate = @@ -2255,28 +2014,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C6: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C6: " "Failed to catch BAD RATE joining an exiting MGID: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Try MTU that does not exist in any MCG */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD MTU (higher them max) when connecting to " "existing MGID (o15.0.1.13)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = osm_ipoib_mgid; mc_req_rec.mtu = IB_MTU_LEN_4096 | IB_PATH_SELECTOR_GREATER_THAN << 6; @@ -2285,28 +2039,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C7: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C7: " "Failed to catch BAD RATE (higher them max) joining an exiting MGID: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Try another MTU that does not exist in any MCG */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD MTU (less than min) when connecting " "to existing MGID (o15.0.1.13)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = osm_ipoib_mgid; mc_req_rec.mtu = IB_MTU_LEN_256 | IB_PATH_SELECTOR_LESS_THAN << 6; @@ -2315,17 +2064,14 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C8: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C8: " "Failed to catch BAD RATE (less them min) joining an exiting MGID: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -2333,8 +2079,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* o15.0.1.14: */ /* - Try partial delete - actually updating the join state. check it. */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking partial JoinState delete request - removing NonMember (o15.0.1.14)...\n"); mc_req_rec.rate = @@ -2350,22 +2095,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02C9: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02C9: " "Fail to partially update JoinState during delete: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating Join State removal of Non Member bit (o15.0.1.14)...\n"); if (p_mc_res->scope_state != 0x25) { /* scope is MSB - now only the non member & send only member have left */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02CA: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02CA: " "Validating JoinState update failed. Expected 0x25 got: 0x%02X\n", p_mc_res->scope_state); status = IB_ERROR; @@ -2379,23 +2120,19 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02CB: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02CB: " "Failed to update JoinState during delete: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Validating Join State update remove (o15.0.1.14)...\n"); if (p_mc_res->scope_state != 0x25) { /* scope is MSB - now only 0x0 so port is removed from MCG */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02BF: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02BF: " "Validating JoinState update failed. Expected 0x25 got: 0x%02X\n", p_mc_res->scope_state); status = IB_ERROR; @@ -2403,36 +2140,29 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) } /* - Try joining (not full mem) again to see the group was deleted. (should fail) */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Delete by trying to Join deleted group (o15.0.1.13)...\n"); mc_req_rec.scope_state = 0x22; /* use non member - so if no group fail */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, /* join */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status != IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02BC: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02BC: " "Succeeded Joining Deleted Group: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* - Try deletion of the IPoIB MCG and get: ERR_REQ_INVALID */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD Delete of Mgid membership (no prev join) (o15.0.1.15)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = osm_ipoib_mgid; mc_req_rec.rate = @@ -2441,27 +2171,23 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 0, /* delete flag */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02BD: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02BD: " "Failed to catch BAD delete from IPoIB: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } /* Prepare another MCG for the following tests : */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Create given MGID=%s\n\t\t(o15.0.1.4)...\n", inet_ntop(AF_INET6, osm_ipoib_mgid.raw, gid_str, - sizeof gid_str)); + sizeof gid_str)); mc_req_rec.mgid = good_mgid; mc_req_rec.mgid.raw[12] = 0xAA; @@ -2473,19 +2199,16 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02BE: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02BE: " "Failed to create MCG for %s - got %s/%s\n", inet_ntop(AF_INET6, good_mgid.raw, gid_str, - sizeof gid_str), - ib_get_err_str(status), + sizeof gid_str), ib_get_err_str(status), ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); goto Exit; } /* - Try delete with valid join state */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Full Delete of a group (o15.0.1.14)...\n"); mc_req_rec.scope_state = 0x21; /* the FullMember is the current JoinState */ status = osmt_send_mcast_request(p_osmt, 0, @@ -2497,11 +2220,9 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* o15.0.1.15: */ /* - Try deletion of the IPoIB MCG and get: ERR_REQ_INVALID */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking BAD Delete of IPoIB membership (no prev join) (o15.0.1.15)...\n"); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); mc_req_rec.mgid = osm_ipoib_mgid; mc_req_rec.rate = @@ -2511,17 +2232,14 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_send_mcast_request(p_osmt, 0, /* delete flag */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if ((status != IB_REMOTE_ERROR) || (res_sa_mad.status != IB_SA_MAD_STATUS_REQ_INVALID)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 0223: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0223: " "Failed to catch BAD delete from IPoIB: %s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -2529,8 +2247,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /**************************************************************************/ /* Checking join with invalid MTU */ osmt_init_mc_query_rec(p_osmt, &mc_req_rec); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Join with unrealistic mtu : \n" "\t\tFirst create new MCG than try to join it \n" "\t\twith unrealistic MTU greater than 4096 (o15.0.1.8)...\n"); @@ -2546,8 +2263,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); p_mc_res = ib_sa_mad_get_payload_ptr(&res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02EB: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02EB: " "Failed to create new mgrp\n"); goto Exit; } @@ -2565,20 +2281,16 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_JOIN_STATE | IB_MCR_COMPMASK_MTU_SEL | IB_MCR_COMPMASK_MTU; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 1, &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02E4: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02E4: " "Expected REMOTE ERROR got:%s/%s\n", ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad)) - ); + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); status = IB_ERROR; goto Exit; } @@ -2586,14 +2298,12 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* - Try GetTable with PortGUID wildcarded and get back some groups. */ status = osmt_query_mcast(p_osmt); cnt = cl_qmap_count(&p_osmt->exp_subn.mgrp_mlid_tbl); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow (Before checking Max MCG creation): " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "(Before checking Max MCG creation): " "Number of MC Records found in SA DB is %d\n", cnt); /**************************************************************************/ /* Checking join on behalf of remote port gid */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " "Checking Proxy Join...\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Proxy Join...\n"); osmt_init_mc_query_rec(p_osmt, &mc_req_rec); memset(&context, 0, sizeof(context)); @@ -2604,8 +2314,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02E5: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02E5: " "osmtest_get_all_recs failed on getting all node records(%s)\n", ib_get_err_str(status)); goto Exit; @@ -2615,17 +2324,15 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) * Populate the database with the received records. */ num_recs = context.result.result_cnt; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "Received %u records\n", num_recs); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { p_rec = osmv_get_query_node_rec(context.result.p_result_madw, i); if (p_rec->node_info.port_guid != p_osmt->local_port.port_guid && p_rec->node_info.node_type == IB_NODE_TYPE_CA) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow: " "remote port_guid = 0x%" - PRIx64 "\n", + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "remote port_guid = 0x%" PRIx64 "\n", cl_ntoh64(p_rec->node_info.port_guid)); remote_port_guid = p_rec->node_info.port_guid; @@ -2646,8 +2353,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02B4: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B4: " "Could not join on behalf of remote port 0x%016" PRIx64 " remote status: %s\n", cl_ntoh64(remote_port_guid), @@ -2662,8 +2368,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* First try a bad deletion then good one */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Trying deletion of remote port with local port guid\n"); osmt_init_mc_query_rec(p_osmt, &mc_req_rec); @@ -2673,19 +2378,16 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_MGID | IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_JOIN_STATE; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmt_send_mcast_request(p_osmt, 0, /* delete flag */ &mc_req_rec, comp_mask, &res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02A9: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02A9: " "Successful deletion of remote port guid with local one MGID : " "%s, Got : %s/%s\n", inet_ntop(AF_INET6, @@ -2698,8 +2400,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Trying deletion of remote port with the right port guid\n"); osmt_init_mc_query_rec(p_osmt, &mc_req_rec); @@ -2714,8 +2415,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02B0: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B0: " "Failed to delete mgid with remote port guid MGID : " "%s, Got : %s/%s\n", inet_ntop(AF_INET6, @@ -2727,8 +2427,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) goto Exit; } } else { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Could not check proxy join since could not found remote port, different from local port\n"); } @@ -2739,8 +2438,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (p_osmt->opt.mmode > 2) { /* Check invalid Join with max mlid which is more than the Mellanox switches support 0xC000+0x1000 = 0xd000 */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Checking Creation of Maximum avaliable Groups (MulticastFDBCap)...\n"); tmp_mlid = cl_ntoh16(max_mlid) - cnt; @@ -2770,8 +2468,8 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (cur_mlid > cl_ntoh16(max_mlid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow : ERR 2E1 " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 2E1 " "Successful Join with greater mlid than switches support (MulticastFDBCap) 0x%04X\n", cur_mlid); status = IB_ERROR; @@ -2793,12 +2491,11 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) p_recvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(&res_sa_mad); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow : " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Created MGID:%s MLID:0x%04X\n", inet_ntop(AF_INET6, - p_recvd_rec->mgid.raw, - gid_str, sizeof gid_str), + p_recvd_rec->mgid.raw, + gid_str, sizeof gid_str), cl_ntoh16(p_recvd_rec->mlid)); cl_map_insert(&test_created_mlids, cl_ntoh16(p_recvd_rec->mlid), @@ -2817,21 +2514,18 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_query_mcast(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02B1: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B1: " "Failed to query multicast groups: %s\n", ib_get_err_str(status)); goto Exit; } cnt = cl_qmap_count(&p_osmt->exp_subn.mgrp_mlid_tbl); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow (Before Deletion of all MCG): " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "(Before Deletion of all MCG): " "Number of MC Records found in SA DB is %d\n", cnt); /* Delete all MCG that are not of IPoIB */ - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow : " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Cleanup all MCG that are not IPoIB...\n"); p_mgrp_mlid_tbl = &p_osmt->exp_subn.mgrp_mlid_tbl; @@ -2856,28 +2550,25 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) IB_MCR_COMPMASK_PORT_GID | IB_MCR_COMPMASK_JOIN_STATE; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_run_mcast_flow : " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Sending request to delete MGID : %s" ", scope_state : 0x%02X\n", inet_ntop(AF_INET6, mc_req_rec.mgid.raw, - gid_str, sizeof gid_str), + gid_str, sizeof gid_str), mc_req_rec.scope_state); status = osmt_send_mcast_request(p_osmt, 0, /* delete flag */ &mc_req_rec, comp_mask, &res_sa_mad); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02FF: " - "Failed to delete MGID : %s" + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 02FF: Failed to delete MGID : %s" " ,\n\t\t it is not our MCG, Status : %s/%s\n", inet_ntop(AF_INET6, - p_mgrp->mcmember_rec.mgid.raw, - gid_str, sizeof gid_str), + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str), ib_get_err_str(status), - ib_get_mad_status_str((ib_mad_t - *) + ib_get_mad_status_str((ib_mad_t *) (&res_sa_mad))); fail_to_delete_mcg++; } @@ -2890,8 +2581,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) status = osmt_query_mcast(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02B2 " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02B2 " "GetTable of all records has failed - got %s\n", ib_get_err_str(status)); goto Exit; @@ -2903,7 +2593,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) if (p_osmt->opt.mmode == 1 || p_osmt->opt.mmode == 3) { end_cnt = cl_qmap_count(&p_osmt->exp_subn.mgrp_mlid_tbl); - osm_log(&p_osmt->log, OSM_LOG_INFO, "osmt_run_mcast_flow: " "Status of MC Records in SA DB during the test flow:\n" " Beginning of test\n" " Unrelated to the test: %d\n" " IPoIB MC Records : %d\n" " Total : %d\n" " End of test\n" " Failed to delete : %d\n" " IPoIB MC Records : %d\n" " Total : %d\n", mcg_outside_test_cnt, /* Non-IPoIB that existed at the beginning */ + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Status of MC Records in SA DB during the test flow:\n" " Beginning of test\n" " Unrelated to the test: %d\n" " IPoIB MC Records : %d\n" " Total : %d\n" " End of test\n" " Failed to delete : %d\n" " IPoIB MC Records : %d\n" " Total : %d\n", mcg_outside_test_cnt, /* Non-IPoIB that existed at the beginning */ start_ipoib_cnt, /* IPoIB records */ start_cnt, /* Total: IPoIB and MC Records unrelated to the test */ fail_to_delete_mcg, /* Failed to delete at the end */ @@ -2913,8 +2603,7 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) /* when we compare num of MCG we should consider an outside source which create other MCGs */ if ((end_cnt - fail_to_delete_mcg - end_ipoib_cnt) != (start_cnt - mcg_outside_test_cnt - start_ipoib_cnt)) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Got different number of non-IPoIB records stored in SA DB\n\t\t" "at Start got %d, at End got %d (IPoIB groups only)\n", (start_cnt - mcg_outside_test_cnt - @@ -2929,25 +2618,22 @@ ib_api_status_t osmt_run_mcast_flow(IN osmtest_t * const p_osmt) uint16_t mlid = (uint16_t) cl_qmap_key((cl_map_item_t *) p_mgrp); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Found MLID:0x%04X\n", mlid); /* Check if the mlid is in the test_created_mlids. If TRUE, then we didn't delete a MCgroup that was created in this flow. */ if (cl_map_get(&test_created_mlids, mlid) != NULL) { /* This means that we still have an mgrp that we created!! */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_mcast_flow: ERR 02FE: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 02FE: " "Wasn't able to erase mgrp with MGID:%s" " MLID:0x%04X\n", inet_ntop(AF_INET6, - p_mgrp->mcmember_rec.mgid.raw, - gid_str, sizeof gid_str), + p_mgrp->mcmember_rec.mgid.raw, + gid_str, sizeof gid_str), mlid); got_error = TRUE; } else { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_run_mcast_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Still exists %s MGID:%s\n", (IS_IPOIB_MGID (&p_mgrp->mcmember_rec. diff --git a/opensm/osmtest/osmt_service.c b/opensm/osmtest/osmt_service.c index 5d3ff8b..ce13500 100644 --- a/opensm/osmtest/osmt_service.c +++ b/opensm/osmtest/osmt_service.c @@ -74,8 +74,7 @@ osmt_register_service(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(p_log); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_register_service: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Registering service: name: %s id: 0x%" PRIx64 "\n", service_name, cl_ntoh64(service_id)); @@ -129,8 +128,7 @@ osmt_register_service(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service: ERR 4A01: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A01: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -138,13 +136,11 @@ osmt_register_service(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service: ERR 4A02: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A02: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -185,8 +181,7 @@ osmt_register_service_with_full_key(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(p_log); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_register_service_with_full_key: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Registering service: name: %s id: 0x%" PRIx64 "\n", service_name, cl_ntoh64(service_id)); @@ -241,21 +236,18 @@ osmt_register_service_with_full_key(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_full_key: ERR 4A03: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A03: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_full_key: ERR 4A04: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A04: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_full_key: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -266,18 +258,17 @@ osmt_register_service_with_full_key(IN osmtest_t * const p_osmt, /* Check service key on context to see if match */ p_rec = osmv_get_query_svc_rec(context.result.p_result_madw, 0); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Comparing service key...\n" "return key is:\n"); for (i = 0; i <= 15; i++) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "service_key sent[%u] = %u, service_key returned[%u] = %u\n", i, service_key[i], i, p_rec->service_key[i]); } /* since c15-0.1.14 not supported all key association queries should bring in return zero in service key */ if (memcmp(skey, p_rec->service_key, 16 * sizeof(uint8_t)) != 0) { status = IB_REMOTE_ERROR; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_full_key: ERR 4A33: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A33: " "Data mismatch in service_key\n"); goto Exit; } @@ -318,8 +309,7 @@ osmt_register_service_with_data(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(p_log); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_register_service_with_data: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Registering service: name: %s id: 0x%" PRIx64 "\n", service_name, cl_ntoh64(service_id)); @@ -395,15 +385,13 @@ osmt_register_service_with_data(IN osmtest_t * const p_osmt, user.p_attr = &svc_rec; /* Dump to Service Data b4 send */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_register_service_with_data:" + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Dumping service data b4 send\n"); osm_dump_service_record(&p_osmt->log, &svc_rec, OSM_LOG_VERBOSE); status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_data: ERR 4A05: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A05: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -411,13 +399,11 @@ osmt_register_service_with_data(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_data: ERR 4A06: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A06: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_data: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -428,7 +414,7 @@ osmt_register_service_with_data(IN osmtest_t * const p_osmt, /* Check data on context to see if match */ p_rec = osmv_get_query_svc_rec(context.result.p_result_madw, 0); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, "Comparing service data...\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Comparing service data...\n"); if (memcmp(service_data8, p_rec->service_data8, 16 * sizeof(uint8_t)) != 0 || memcmp(service_data16, p_rec->service_data16, @@ -438,8 +424,7 @@ osmt_register_service_with_data(IN osmtest_t * const p_osmt, || memcmp(service_data64, p_rec->service_data64, 2 * sizeof(uint64_t)) != 0) { status = IB_REMOTE_ERROR; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_register_service_with_data: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Data mismatch in service_data8\n"); goto Exit; } @@ -475,11 +460,9 @@ osmt_get_service_by_id_and_name(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_id_and_name: " - "Getting service record: id: 0x%016" PRIx64 - " and name: %s\n", cl_ntoh64(sid), sr_name); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting service record: id: 0x%016" PRIx64 + " and name: %s\n", cl_ntoh64(sid), sr_name); /* * Do a blocking query for this record in the subnet. @@ -519,8 +502,7 @@ osmt_get_service_by_id_and_name(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id_and_name: ERR 4A07: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A07: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -543,13 +525,11 @@ osmt_get_service_by_id_and_name(IN osmtest_t * const p_osmt, if (status == IB_REMOTE_ERROR && !strcmp(mad_stat_err, "IB_SA_MAD_STATUS_NO_RECORDS") && rec_num == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id_and_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "IS EXPECTED ERROR ^^^^\n"); status = IB_SUCCESS; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id_and_name: ERR 4A08: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A08: " "Query failed: %s (%s)\n", ib_get_err_str(status), mad_stat_err); goto Exit; @@ -557,8 +537,7 @@ osmt_get_service_by_id_and_name(IN osmtest_t * const p_osmt, } if (rec_num && num_recs != rec_num) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id_and_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Unmatched number of records: expected: %d, received: %d\n", rec_num, num_recs); status = IB_REMOTE_ERROR; @@ -569,8 +548,7 @@ osmt_get_service_by_id_and_name(IN osmtest_t * const p_osmt, *p_out_rec = *p_rec; if (num_recs) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_id_and_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Found service record: name: %s id: 0x%016" PRIx64 "\n", p_rec->service_name, cl_ntoh64(p_rec->service_id)); @@ -578,8 +556,7 @@ osmt_get_service_by_id_and_name(IN osmtest_t * const p_osmt, } Exit: - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_id_and_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Expected and found %d records\n", rec_num); if (context.result.p_result_madw != NULL) { @@ -610,11 +587,9 @@ osmt_get_service_by_id(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_id: " - "Getting service record: id: 0x%016" PRIx64 "\n", - cl_ntoh64(sid)); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting service record: id: 0x%016" PRIx64 "\n", + cl_ntoh64(sid)); /* * Do a blocking query for this record in the subnet. @@ -651,8 +626,7 @@ osmt_get_service_by_id(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id: ERR 4A09: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A09: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -676,13 +650,11 @@ osmt_get_service_by_id(IN osmtest_t * const p_osmt, if (status == IB_REMOTE_ERROR && !strcmp(mad_stat_err, "IB_SA_MAD_STATUS_NO_RECORDS") && rec_num == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "IS EXPECTED ERROR ^^^^\n"); status = IB_SUCCESS; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id: ERR 4A0A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A0A: " "Query failed: %s (%s)\n", ib_get_err_str(status), mad_stat_err); goto Exit; @@ -690,8 +662,7 @@ osmt_get_service_by_id(IN osmtest_t * const p_osmt, } if (rec_num && num_recs != rec_num) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_id: ERR 4A0B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A0B: " "Unmatched number of records: expected: %d received: %d\n", rec_num, num_recs); status = IB_REMOTE_ERROR; @@ -702,8 +673,7 @@ osmt_get_service_by_id(IN osmtest_t * const p_osmt, *p_out_rec = *p_rec; if (num_recs) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_id: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Found service record: name: %s id: 0x%016" PRIx64 "\n", p_rec->service_name, cl_ntoh64(p_rec->service_id)); @@ -711,8 +681,7 @@ osmt_get_service_by_id(IN osmtest_t * const p_osmt, } Exit: - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_id: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Expected and found %d records\n", rec_num); if (context.result.p_result_madw != NULL) { @@ -745,20 +714,12 @@ osmt_get_service_by_name_and_key(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - char buf_service_key[33]; - - sprintf(buf_service_key, - "0x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x", - skey[0], skey[1], skey[2], skey[3], skey[4], skey[5], - skey[6], skey[7], skey[8], skey[9], skey[10], skey[11], - skey[12], skey[13], skey[14], skey[15]); - - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_name_and_key: " - "Getting service record: name: %s and key: %s\n", - sr_name, buf_service_key); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting service record: name: %s and key: " + "0x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x\n", + sr_name, skey[0], skey[1], skey[2], skey[3], skey[4], skey[5], + skey[6], skey[7], skey[8], skey[9], skey[10], skey[11], + skey[12], skey[13], skey[14], skey[15]); /* * Do a blocking query for this record in the subnet. @@ -799,8 +760,7 @@ osmt_get_service_by_name_and_key(IN osmtest_t * const p_osmt, user.p_attr = &svc_rec; status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name_and_key: ERR 4A0C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A0C: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -824,13 +784,11 @@ osmt_get_service_by_name_and_key(IN osmtest_t * const p_osmt, if (status == IB_REMOTE_ERROR && !strcmp(mad_stat_err, "IB_SA_MAD_STATUS_NO_RECORDS") && rec_num == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name_and_key: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "IS EXPECTED ERROR ^^^^\n"); status = IB_SUCCESS; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name_and_key: ERR 4A0D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A0D: " "Query failed:%s (%s)\n", ib_get_err_str(status), mad_stat_err); goto Exit; @@ -838,8 +796,7 @@ osmt_get_service_by_name_and_key(IN osmtest_t * const p_osmt, } if (rec_num && num_recs != rec_num) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name_and_key: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Unmatched number of records: expected: %d, received: %d\n", rec_num, num_recs); status = IB_REMOTE_ERROR; @@ -850,8 +807,7 @@ osmt_get_service_by_name_and_key(IN osmtest_t * const p_osmt, *p_out_rec = *p_rec; if (num_recs) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_name_and_key: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Found service record: name: %s id: 0x%016" PRIx64 "\n", sr_name, cl_ntoh64(p_rec->service_id)); @@ -859,8 +815,7 @@ osmt_get_service_by_name_and_key(IN osmtest_t * const p_osmt, } Exit: - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_name_and_key: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Expected and found %d records\n", rec_num); if (context.result.p_result_madw != NULL) { @@ -892,10 +847,8 @@ osmt_get_service_by_name(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_name: " - "Getting service record: name: %s\n", sr_name); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting service record: name: %s\n", sr_name); /* * Do a blocking query for this record in the subnet. @@ -924,8 +877,7 @@ osmt_get_service_by_name(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name: ERR 4A0E: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A0E: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -949,13 +901,11 @@ osmt_get_service_by_name(IN osmtest_t * const p_osmt, if (status == IB_REMOTE_ERROR && !strcmp(mad_stat_err, "IB_SA_MAD_STATUS_NO_RECORDS") && rec_num == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "IS EXPECTED ERROR ^^^^\n"); status = IB_SUCCESS; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name: ERR 4A0F: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A0F: " "Query failed: %s (%s)\n", ib_get_err_str(status), mad_stat_err); goto Exit; @@ -963,8 +913,7 @@ osmt_get_service_by_name(IN osmtest_t * const p_osmt, } if (rec_num && num_recs != rec_num) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_service_by_name: ERR 4A10: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A10: " "Unmatched number of records: expected: %d, received: %d\n", rec_num, num_recs); status = IB_REMOTE_ERROR; @@ -975,8 +924,7 @@ osmt_get_service_by_name(IN osmtest_t * const p_osmt, *p_out_rec = *p_rec; if (num_recs) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Found service record: name: %s id: 0x%016" PRIx64 "\n", sr_name, cl_ntoh64(p_rec->service_id)); @@ -984,8 +932,7 @@ osmt_get_service_by_name(IN osmtest_t * const p_osmt, } Exit: - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_service_by_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Expected and found %d records\n", rec_num); if (context.result.p_result_madw != NULL) { @@ -1025,11 +972,8 @@ osmt_get_all_services_and_check_names(IN osmtest_t * const p_osmt, p_checked_names[j] = 0; } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_all_services_and_check_names: " - "Getting all service records\n"); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Getting all service records\n"); + /* * Do a blocking query for this record in the subnet. * The result is returned in the result field of the caller's @@ -1052,8 +996,7 @@ osmt_get_all_services_and_check_names(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_all_services_and_check_names: ERR 4A12: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A12: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -1062,14 +1005,11 @@ osmt_get_all_services_and_check_names(IN osmtest_t * const p_osmt, if (status != IB_SUCCESS) { if (status != IB_INVALID_PARAMETER) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_all_services_and_check_names: ERR 4A13: " - "ib_query failed (%s)\n", - ib_get_err_str(status)); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A13: " + "ib_query failed (%s)\n", ib_get_err_str(status)); } if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_all_services_and_check_names: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -1079,29 +1019,25 @@ osmt_get_all_services_and_check_names(IN osmtest_t * const p_osmt, } num_recs = context.result.result_cnt; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_all_services_and_check_names: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { p_rec = osmv_get_query_svc_rec(context.result.p_result_madw, i); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_all_services_and_check_names: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Found service record: name: %s id: 0x%016" PRIx64 "\n", p_rec->service_name, cl_ntoh64(p_rec->service_id)); osm_dump_service_record(&p_osmt->log, p_rec, OSM_LOG_VERBOSE); for (j = 0; j < num_of_valid_names; j++) { /* If the service names exist in the record, mark it as checked (1) */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_all_services_and_check_names: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "-I- Comparing source name : >%s<, with record name : >%s<, idx : %d\n", p_valid_service_names_arr[j], p_rec->service_name, p_checked_names[j]); if (strcmp ((char *)p_valid_service_names_arr[j], (char *)p_rec->service_name) == 0) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_get_all_services_and_check_names: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "-I- The service %s is valid\n", p_valid_service_names_arr[j]); p_checked_names[j] = 1; @@ -1112,8 +1048,7 @@ osmt_get_all_services_and_check_names(IN osmtest_t * const p_osmt, /* Check that all service names have been identified */ for (j = 0; j < num_of_valid_names; j++) if (p_checked_names[j] == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_get_all_services_and_check_names: ERR 4A14: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A14: " "Missing valid service: name: %s\n", p_valid_service_names_arr[j]); status = IB_ERROR; @@ -1149,16 +1084,14 @@ osmt_delete_service_by_name(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_delete_service_by_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Trying to Delete service name: %s\n", sr_name); memset(&svc_rec, 0, sizeof(svc_rec)); status = osmt_get_service_by_name(p_osmt, sr_name, rec_num, &svc_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_delete_service_by_name: ERR 4A15: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A15: " "Failed to get service: name: %s\n", sr_name); goto ExitNoDel; } @@ -1190,8 +1123,7 @@ osmt_delete_service_by_name(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_delete_service_by_name: ERR 4A16: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A16: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -1200,15 +1132,13 @@ osmt_delete_service_by_name(IN osmtest_t * const p_osmt, if (IsServiceExist) { /* If IsServiceExist = 1 then we should succeed here */ if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_delete_service_by_name: ERR 4A17: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A17: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_delete_service_by_name: ERR 4A18: " - "Remote error = %s\n", + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 4A18: Remote error = %s\n", ib_get_mad_status_str (osm_madw_get_mad_ptr (context.result.p_result_madw))); @@ -1217,19 +1147,16 @@ osmt_delete_service_by_name(IN osmtest_t * const p_osmt, } else { /* If IsServiceExist = 0 then we should fail here */ if (status == IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_delete_service_by_name: ERR 4A19: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A19: " "Succeeded to delete service: %s which " "shouldn't exist", sr_name); status = IB_ERROR; } else { /* The deletion should have failed, since the service_name shouldn't exist. */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "IS EXPECTED ERROR ^^^^\n"); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmt_delete_service_by_name: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "Failed to delete service_name: %s\n", sr_name); status = IB_SUCCESS; } @@ -1386,8 +1313,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[0], 1, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A1A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A1A: " "Fail to find service: name: %s\n", (char *)service_name[0]); status = IB_ERROR; @@ -1398,8 +1324,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[1], 1, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A1B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A1B: " "Fail to find service: name: %s\n", (char *)service_name[1]); status = IB_ERROR; @@ -1410,8 +1335,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[2], 1, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A1C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A1C: " "Fail to find service: name: %s\n", (char *)service_name[2]); status = IB_ERROR; @@ -1423,8 +1347,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[3], 1, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A1D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A1D: " "Fail to find service: name: %s\n", (char *)service_name[3]); status = IB_ERROR; @@ -1436,8 +1359,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[3], 0, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A1E: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A1E: " "Found service: name: %s that should have been " "deleted due to service lease expiring\n", (char *)service_name[3]); @@ -1448,8 +1370,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) /* Check that for service: id[5] only one record exists */ status = osmt_get_service_by_id(p_osmt, 1, cl_ntoh64(id[5]), &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A1F: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A1F: " "Found number of records != 1 for " "service: id: 0x%016" PRIx64 "\n", id[5]); status = IB_ERROR; @@ -1459,8 +1380,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) /* Bad Flow of Get with invalid Service ID: id[7] */ status = osmt_get_service_by_id(p_osmt, 0, cl_ntoh64(id[7]), &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A20: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A20: " "Found service: id: 0x%016" PRIx64 " " "that is invalid\n", id[7]); status = IB_ERROR; @@ -1472,8 +1392,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[0], &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A21: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A21: " "Fail to find service: id: 0x%016" PRIx64 " " "name: %s\n", id[0], (char *)service_name[0]); status = IB_ERROR; @@ -1485,8 +1404,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[6], &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A22: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A22: " "Fail to find service: id: 0x%016" PRIx64 " " "name: %s\n", id[5], (char *)service_name[6]); status = IB_ERROR; @@ -1498,8 +1416,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[3], &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A23: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A23: " "Found service: id: 0x%016" PRIx64 "name: %s which is an invalid service\n", id[0], (char *)service_name[3]); @@ -1512,8 +1429,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[5], &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A24: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A24: " "Found service: id: 0x%016" PRIx64 "name: %s which is an invalid service\n", id[3], (char *)service_name[5]); @@ -1525,8 +1441,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[4], 0, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A25: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A25: " "Found service: name: %s that shouldn't exist\n", (char *)service_name[4]); status = IB_ERROR; @@ -1538,8 +1453,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[5], 0, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A26: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A26: " "Found service: name: %s which is an " "invalid service\n", (char *)service_name[5]); status = IB_ERROR; @@ -1551,8 +1465,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[6], 1, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A27: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A27: " "Fail to find service: name: %s\n", (char *)service_name[6]); status = IB_ERROR; @@ -1568,8 +1481,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[5], 0, service_key, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A28: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A28: " "Found service: name: %s key:0 which is an " "invalid service (wrong name)\n", (char *)service_name[5]); @@ -1583,8 +1495,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[6], 0, service_key, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A29: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A29: " "Found service: name: %s key: 0 which is an " "invalid service (wrong service_key)\n", (char *)service_name[6]); @@ -1599,8 +1510,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) (char *)service_name[6], 1, service_key, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A2A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A2A: " "Fail to find service: name: %s with " "correct service key\n", (char *)service_name[6]); status = IB_ERROR; @@ -1616,8 +1526,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) osmt_get_all_services_and_check_names(p_osmt, service_valid_names, 3, &num_recs); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A2B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A2B: " "Fail to find all services that should exist\n"); status = IB_ERROR; goto Exit; @@ -1628,8 +1537,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_delete_service_by_name(p_osmt, 1, (char *)service_name[0], 1); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A2C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A2C: " "Fail to delete service: name: %s\n", (char *)service_name[0]); status = IB_ERROR; @@ -1640,8 +1548,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[0], 0, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A2D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A2D: " "Found service: name: %s that was deleted\n", (char *)service_name[0]); status = IB_ERROR; @@ -1652,8 +1559,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[1], 0, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A2E: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A2E: " "Found service: name: %s that should have expired\n", (char *)service_name[1]); status = IB_ERROR; @@ -1664,8 +1570,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_get_service_by_name(p_osmt, (char *)service_name[2], 1, &srv_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A2F: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A2F: " "Fail to find service: name: %s\n", (char *)service_name[2]); status = IB_ERROR; @@ -1676,8 +1581,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_delete_service_by_name(p_osmt, 0, (char *)service_name[5], 0); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A30: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A30: " "Succeed to delete non-existent service: name: %s\n", (char *)service_name[5]); status = IB_ERROR; @@ -1688,8 +1592,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_delete_service_by_name(p_osmt, 1, (char *)service_name[2], 1); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A31: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A31: " "Fail to delete service: name: %s\n", (char *)service_name[2]); status = IB_ERROR; @@ -1700,8 +1603,7 @@ ib_api_status_t osmt_run_service_records_flow(IN osmtest_t * const p_osmt) status = osmt_delete_service_by_name(p_osmt, 1, (char *)service_name[6], 1); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_run_service_records_flow: ERR 4A32: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 4A32: " "Failed to delete service name: %s\n", (char *)service_name[6]); goto Exit; diff --git a/opensm/osmtest/osmt_slvl_vl_arb.c b/opensm/osmtest/osmt_slvl_vl_arb.c index df234d2..e63bef2 100644 --- a/opensm/osmtest/osmt_slvl_vl_arb.c +++ b/opensm/osmtest/osmt_slvl_vl_arb.c @@ -106,12 +106,9 @@ osmt_query_vl_arb(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmt_query_vl_arb: " - "Getting VL_Arbitration Table for port with LID 0x%X Num:0x%X\n", - cl_ntoh16(lid), port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Getting VL_Arbitration Table for port with LID 0x%X Num:0x%X\n", + cl_ntoh16(lid), port_num); /* * Do a blocking query for this record in the subnet. @@ -143,8 +140,7 @@ osmt_query_vl_arb(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_vl_arb: ERR 0405: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0405: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -152,13 +148,11 @@ osmt_query_vl_arb(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_vl_arb: ERR 0466: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0466: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_vl_arb: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -197,8 +191,7 @@ osmt_query_all_ports_vl_arb(IN osmtest_t * const p_osmt, IN FILE * fh) OSM_LOG_ENTER(&p_osmt->log); - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_query_all_ports_vl_arb: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Obtaining ALL Ports VL Arbitration Tables\n"); /* @@ -223,8 +216,8 @@ osmt_query_all_ports_vl_arb(IN osmtest_t * const p_osmt, IN FILE * fh) p_src_port->rec.lid, 0, block, fh); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_all_ports_vl_arb: ERR 0467: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0467: " "Failed to get Lid:0x%X Port:0x%X (%s)\n", cl_ntoh16(p_src_port->rec.lid), 0, ib_get_err_str(status)); @@ -240,8 +233,8 @@ osmt_query_all_ports_vl_arb(IN osmtest_t * const p_osmt, IN FILE * fh) p_src_port->rec.port_num, block, fh); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_all_ports_vl_arb: ERR 0468: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0468: " "Failed to get Lid:0x%X Port:0x%X (%s)\n", cl_ntoh16(p_src_port->rec.lid), p_src_port->rec.port_num, @@ -314,12 +307,9 @@ osmt_query_slvl_map(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmt_query_slvl_map: " - "Getting SLtoVL Map Table for out-port with LID 0x%X Num:0x%X from In-Port:0x%X\n", - cl_ntoh16(lid), out_port_num, in_port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Getting SLtoVL Map Table for out-port with LID 0x%X Num:0x%X from In-Port:0x%X\n", + cl_ntoh16(lid), out_port_num, in_port_num); /* * Do a blocking query for this record in the subnet. @@ -351,8 +341,7 @@ osmt_query_slvl_map(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_slvl_map: ERR 0469: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0469: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -360,13 +349,11 @@ osmt_query_slvl_map(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_slvl_map: ERR 0470: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0470: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_slvl_map: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -412,8 +399,7 @@ osmt_query_all_ports_slvl_map(IN osmtest_t * const p_osmt, IN FILE * fh) * get the relevant SLtoVLs */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmt_query_all_ports_slvl_map: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Obtaining ALL Ports (to other ports) SLtoVL Maps\n"); p_tbl = &p_osmt->exp_subn.port_key_tbl; @@ -432,8 +418,7 @@ osmt_query_all_ports_slvl_map(IN osmtest_t * const p_osmt, IN FILE * fh) osmt_query_slvl_map(p_osmt, p_src_port->rec.lid, 0, 0, fh); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_all_ports_slvl_map: ERR 0471: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0471: " "Failed to get Lid:0x%X In-Port:0x%X Out-Port:0x%X(%s)\n", cl_ntoh16(p_src_port->rec.lid), 0, 0, ib_get_err_str(status)); @@ -446,8 +431,7 @@ osmt_query_all_ports_slvl_map(IN osmtest_t * const p_osmt, IN FILE * fh) (node_t *) cl_qmap_get(p_node_tbl, p_src_port->rec.lid); if (p_node == (node_t *) cl_qmap_end(p_node_tbl)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_all_ports_slvl_map: ERR 0472: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0472: " "Failed to get Node by Lid:0x%X\n", p_src_port->rec.lid); goto Exit; @@ -462,8 +446,8 @@ osmt_query_all_ports_slvl_map(IN osmtest_t * const p_osmt, IN FILE * fh) p_src_port->rec. port_num, in_port, fh); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmt_query_all_ports_slvl_map: ERR 0473: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0473: " "Failed to get Lid:0x%X In-Port:0x%X Out-Port:0x%X (%s)\n", cl_ntoh16(p_src_port->rec.lid), p_src_port->rec.port_num, diff --git a/opensm/osmtest/osmtest.c b/opensm/osmtest/osmtest.c index 4e73ed0..b173e49 100644 --- a/opensm/osmtest/osmtest.c +++ b/opensm/osmtest/osmtest.c @@ -507,7 +507,7 @@ osmtest_init(IN osmtest_t * const p_osmt, /* but we do not want any extra stuff here */ osm_log_set_level(&p_osmt->log, log_flags); - osm_log(&p_osmt->log, OSM_LOG_FUNCS, "osmtest_init: [\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_FUNCS, "[\n"); p_osmt->opt = *p_opt; @@ -526,8 +526,7 @@ osmtest_init(IN osmtest_t * const p_osmt, if (p_osmt->p_vendor == NULL) { status = IB_INSUFFICIENT_RESOURCES; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_init: ERR 0001: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0001: " "Unable to allocate vendor object"); status = IB_ERROR; goto Exit; @@ -539,7 +538,7 @@ osmtest_init(IN osmtest_t * const p_osmt, goto Exit; Exit: - osm_log(&p_osmt->log, OSM_LOG_FUNCS, "osmtest_init: ]\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_FUNCS, "]\n"); return (status); } @@ -556,8 +555,7 @@ void osmtest_query_res_cb(IN osmv_query_res_t * p_rec) p_ctxt->result = *p_rec; if (p_rec->status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_query_res_cb: ERR 0003: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0003: " "Error on query (%s)\n", ib_get_err_str(p_rec->status)); } @@ -578,12 +576,8 @@ osmtest_get_all_recs(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_get_all_recs: " - "Getting all %s records\n", - ib_get_sa_attr_str(attr_id)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "Getting all %s records\n", + ib_get_sa_attr_str(attr_id)); /* * Do a blocking query for all records in the subnet. @@ -610,8 +604,7 @@ osmtest_get_all_recs(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_all_recs: ERR 0004: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0004: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -619,13 +612,11 @@ osmtest_get_all_recs(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_all_recs: ERR 0064: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0064: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_all_recs: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (p_context->result. @@ -652,11 +643,7 @@ ib_api_status_t osmtest_validate_sa_class_port_info(IN osmtest_t * const p_osmt) OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_sa_class_port_info: " - "Getting ClassPortInfo\n"); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Getting ClassPortInfo\n"); /* * Do a blocking query for this record in the subnet. @@ -679,8 +666,7 @@ ib_api_status_t osmtest_validate_sa_class_port_info(IN osmtest_t * const p_osmt) status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_sa_class_port_info: ERR 0065: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0065: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -688,12 +674,10 @@ ib_api_status_t osmtest_validate_sa_class_port_info(IN osmtest_t * const p_osmt) status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_sa_class_port_info: ERR 0070: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0070: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_sa_class_port_info: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (p_context->result. @@ -708,9 +692,8 @@ ib_api_status_t osmtest_validate_sa_class_port_info(IN osmtest_t * const p_osmt) p_cpi = (ib_class_port_info_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_madp); - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmtest_validate_sa_class_port_info:\n" - "-----------------------------\n" + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "\n-----------------------------\n" "SA Class Port Info:\n" " base_ver:%u\n" " class_ver:%u\n" @@ -748,12 +731,9 @@ osmtest_get_node_rec(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_node_rec: " - "Getting node record for 0x%016" PRIx64 "\n", - cl_ntoh64(node_guid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting node record for 0x%016" PRIx64 "\n", + cl_ntoh64(node_guid)); /* * Do a blocking query for this record in the subnet. @@ -785,8 +765,7 @@ osmtest_get_node_rec(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_node_rec: ERR 0071: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0071: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -794,12 +773,10 @@ osmtest_get_node_rec(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_node_rec: ERR 0072: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0072: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_node_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (p_context->result. @@ -829,11 +806,8 @@ osmtest_get_node_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_node_rec_by_lid: " - "Getting node record for LID 0x%02X\n", cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting node record for LID 0x%02X\n", cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -865,8 +839,7 @@ osmtest_get_node_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_node_rec_by_lid: ERR 0073: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0073: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -874,15 +847,13 @@ osmtest_get_node_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_node_rec_by_lid: ERR 0074: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0074: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_node_rec_by_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -929,15 +900,13 @@ osmtest_get_path_rec_by_guid_pair(IN osmtest_t * const p_osmt, req.p_query_input = &guid_pair; req.sm_key = 0; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_path_rec_by_guid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Query for path from 0x%" PRIx64 " to 0x%" PRIx64 "\n", sguid, dguid); status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_guid_pair: ERR 0063: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0063: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -945,13 +914,11 @@ osmtest_get_path_rec_by_guid_pair(IN osmtest_t * const p_osmt, status = (*p_context).result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_guid_pair: ERR 0066: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0066: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_guid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr ((*p_context).result. @@ -998,8 +965,7 @@ osmtest_get_path_rec_by_gid_pair(IN osmtest_t * const p_osmt, req.p_query_input = &gid_pair; req.sm_key = 0; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_path_rec_by_gid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Query for path from 0x%016" PRIx64 " 0x%016" PRIx64 " to 0x%016" PRIx64 " 0x%016" PRIx64 "\n", sgid.unicast.prefix, sgid.unicast.interface_id, dgid.unicast.prefix, @@ -1007,8 +973,7 @@ osmtest_get_path_rec_by_gid_pair(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_gid_pair: ERR 006A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 006A: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -1016,13 +981,11 @@ osmtest_get_path_rec_by_gid_pair(IN osmtest_t * const p_osmt, status = (*p_context).result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_gid_pair: ERR 006B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 006B: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_gid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr ((*p_context).result. @@ -1073,8 +1036,7 @@ osmtest_get_multipath_rec(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: ERR 0068: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0068: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -1082,13 +1044,11 @@ osmtest_get_multipath_rec(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: ERR 0069: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0069: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (p_context->result. @@ -1117,12 +1077,9 @@ osmtest_get_port_rec(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_get_port_rec: " - "Getting PortInfoRecord for port with LID 0x%X\n", - cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Getting PortInfoRecord for port with LID 0x%X\n", + cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -1154,8 +1111,7 @@ osmtest_get_port_rec(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_port_rec: ERR 0075: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0075: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -1163,13 +1119,11 @@ osmtest_get_port_rec(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_port_rec: ERR 0076: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0076: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_port_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (p_context->result. @@ -1199,12 +1153,9 @@ osmtest_get_port_rec_by_num(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_get_port_rec_by_num: " - "Getting PortInfoRecord for port with LID 0x%X Num:0x%X\n", - cl_ntoh16(lid), port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Getting PortInfoRecord for port with LID 0x%X Num:0x%X\n", + cl_ntoh16(lid), port_num); /* * Do a blocking query for this record in the subnet. @@ -1234,8 +1185,7 @@ osmtest_get_port_rec_by_num(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_port_rec_by_num: ERR 0077: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0077: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -1243,16 +1193,14 @@ osmtest_get_port_rec_by_num(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_port_rec_by_num: ERR 0078: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0078: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_port_rec_by_num: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); status = @@ -1289,8 +1237,7 @@ osmtest_stress_port_recs_large(IN osmtest_t * const p_osmt, sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_stress_port_recs_large: ERR 0006: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0006: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -1304,8 +1251,7 @@ osmtest_stress_port_recs_large(IN osmtest_t * const p_osmt, ++*p_num_queries; if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_stress_port_recs_large: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { @@ -1355,8 +1301,7 @@ osmtest_stress_node_recs_large(IN osmtest_t * const p_osmt, sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_stress_node_recs_large: ERR 0007: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0007: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -1370,8 +1315,7 @@ osmtest_stress_node_recs_large(IN osmtest_t * const p_osmt, ++*p_num_queries; if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_stress_node_recs_large: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { @@ -1420,8 +1364,7 @@ osmtest_stress_path_recs_large(IN osmtest_t * const p_osmt, status = osmtest_get_all_recs(p_osmt, IB_MAD_ATTR_PATH_RECORD, sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_stress_path_recs_large: ERR 0008: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0008: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -1435,8 +1378,7 @@ osmtest_stress_path_recs_large(IN osmtest_t * const p_osmt, ++*p_num_queries; if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_stress_path_recs_large: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { @@ -1498,8 +1440,7 @@ osmtest_stress_path_recs_by_guid(IN osmtest_t * const p_osmt, /* * Do a blocking query for CA to CA Path Record */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_stress_path_recs_by_guid:" + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Source : guid = 0x%" PRIx64 " type = %d" "Target : guid = 0x%" PRIx64 " type = %d\n", cl_ntoh64(p_src_node->rec.node_info.port_guid), @@ -1525,8 +1466,8 @@ osmtest_stress_path_recs_by_guid(IN osmtest_t * const p_osmt, /* In a case of TIMEOUT you still can try sending but cant count, maybe its a temporary issue */ if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_stress_path_recs_by_guid: ERR 0009: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0009: " "osmtest_get_path_rec_by_guid_pair failed (%s)\n", ib_get_err_str(status)); if (status != IB_TIMEOUT) @@ -1539,13 +1480,8 @@ osmtest_stress_path_recs_by_guid(IN osmtest_t * const p_osmt, */ *p_num_recs += num_recs; ++*p_num_queries; - if (osm_log_is_active - (&p_osmt->log, OSM_LOG_VERBOSE)) - osm_log(&p_osmt->log, - OSM_LOG_VERBOSE, - "osmtest_stress_path_recs_by_guid: " - "Received %u records\n", - num_recs); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Received %u records\n", num_recs); /* Dont waste time if not VERBOSE and above */ if (p_osmt->log.level & OSM_LOG_VERBOSE) { for (i = 0; i < num_recs; i++) { @@ -1605,8 +1541,7 @@ osmtest_stress_port_recs_small(IN osmtest_t * const p_osmt, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_stress_port_recs_small: ERR 0010: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0010: " "osmtest_get_port_rec failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -1620,8 +1555,7 @@ osmtest_stress_port_recs_small(IN osmtest_t * const p_osmt, ++*p_num_queries; if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_stress_port_recs_small: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { @@ -1669,8 +1603,7 @@ osmtest_get_local_port_lmc(IN osmtest_t * const p_osmt, status = osmtest_get_port_rec(p_osmt, cl_ntoh16(lid), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_local_port_lmc: ERR 001A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 001A: " "osmtest_get_port_rec failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -1678,11 +1611,7 @@ osmtest_get_local_port_lmc(IN osmtest_t * const p_osmt, num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_local_port_lmc: " - "Received %u records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %u records\n", num_recs); for (i = 0; i < num_recs; i++) { p_rec = @@ -1691,9 +1620,7 @@ osmtest_get_local_port_lmc(IN osmtest_t * const p_osmt, osm_dump_portinfo_record(&p_osmt->log, p_rec, OSM_LOG_VERBOSE); if (p_lmc) { *p_lmc = ib_port_info_get_lmc(&p_rec->port_info); - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_get_local_port_lmc: " - "LMC %d\n", *p_lmc); + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "LMC %d\n", *p_lmc); } } @@ -1727,12 +1654,9 @@ ib_api_status_t osmtest_wrong_sm_key_ignored(IN osmtest_t * const p_osmt) OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_INFO)) { - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmtest_wrong_sm_key_ignored: " - "Trying PortInfoRecord for port with LID 0x%X Num:0x%X\n", - p_osmt->local_port.sm_lid, port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "Trying PortInfoRecord for port with LID 0x%X Num:0x%X\n", + p_osmt->local_port.sm_lid, port_num); /* * Do a blocking query for this record in the subnet. @@ -1761,16 +1685,13 @@ ib_api_status_t osmtest_wrong_sm_key_ignored(IN osmtest_t * const p_osmt) req.sm_key = 9999; context.result.p_result_madw = NULL; - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_wrong_sm_key_ignored: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmv_query_sa(p_osmt->h_bind, &req); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_wrong_sm_key_ignored: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); /* since we use a wrong sm_key we should get a timeout */ if (status != IB_TIMEOUT) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_wrong_sm_key_ignored: ERR 0011: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0011: " "Did not get a timeout but got (%s)\n", ib_get_err_str(status)); if (status == IB_SUCCESS) { @@ -1878,8 +1799,8 @@ osmtest_write_port_info(IN osmtest_t * const p_osmt, p_rec->port_info.error_threshold); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_port_info: ERR 0161: " "Write failed\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0161: " + "Write failed\n"); status = IB_ERROR; goto Exit; } @@ -1928,8 +1849,8 @@ osmtest_write_path_info(IN osmtest_t * const p_osmt, p_rec->preference); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_path_info: ERR 0162: " "Write failed\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0162: " + "Write failed\n"); status = IB_ERROR; goto Exit; } @@ -1988,8 +1909,8 @@ osmtest_write_node_info(IN osmtest_t * const p_osmt, (&p_rec->node_info)), desc); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_node_info: ERR 0163: " "Write failed\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0163: " + "Write failed\n"); status = IB_ERROR; goto Exit; } @@ -2022,8 +1943,8 @@ osmtest_write_link(IN osmtest_t * const p_osmt, p_rec->to_port_num, cl_ntoh16(p_rec->to_lid)); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_link: ERR 0164: " "Write failed\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0164: " + "Write failed\n"); status = IB_ERROR; goto Exit; } @@ -2056,8 +1977,7 @@ osmtest_write_all_link_recs(IN osmtest_t * const p_osmt, IN FILE * fh) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_link_recs: ERR 0165: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0165: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -2068,16 +1988,12 @@ osmtest_write_all_link_recs(IN osmtest_t * const p_osmt, IN FILE * fh) */ num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_write_all_link_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Received %zu records\n", num_recs); result = fprintf(fh, "#\n" "# Link Records\n" "#\n"); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_link_recs: ERR 0166: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0166: " "Write failed\n"); status = IB_ERROR; goto Exit; @@ -2138,13 +2054,11 @@ osmtest_get_path_rec_by_lid_pair(IN osmtest_t * const p_osmt, req.p_query_input = &lid_pair; req.sm_key = 0; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Query for path from 0x%X to 0x%X\n", slid, dlid); status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: ERR 0053: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0053: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -2152,13 +2066,11 @@ osmtest_get_path_rec_by_lid_pair(IN osmtest_t * const p_osmt, status = (*p_context).result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: ERR 0067: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0067: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr ((*p_context).result. @@ -2197,8 +2109,7 @@ osmtest_write_all_node_recs(IN osmtest_t * const p_osmt, IN FILE * fh) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_node_recs: ERR 0022: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0022: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -2209,16 +2120,11 @@ osmtest_write_all_node_recs(IN osmtest_t * const p_osmt, IN FILE * fh) */ num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_write_all_node_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %zu records\n", num_recs); result = fprintf(fh, "#\n" "# Node Records\n" "#\n"); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_node_recs: ERR 0023: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0023: " "Write failed\n"); status = IB_ERROR; goto Exit; @@ -2268,8 +2174,7 @@ osmtest_write_all_port_recs(IN osmtest_t * const p_osmt, IN FILE * fh) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_port_recs: ERR 0167: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0167: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -2280,16 +2185,11 @@ osmtest_write_all_port_recs(IN osmtest_t * const p_osmt, IN FILE * fh) */ num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_write_all_port_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %zu records\n", num_recs); result = fprintf(fh, "#\n" "# PortInfo Records\n" "#\n"); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_port_recs: ERR 0024: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0024: " "Write failed\n"); status = IB_ERROR; goto Exit; @@ -2340,8 +2240,7 @@ osmtest_write_all_path_recs(IN osmtest_t * const p_osmt, IN FILE * fh) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_path_recs: ERR 0025: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0025: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -2352,16 +2251,11 @@ osmtest_write_all_path_recs(IN osmtest_t * const p_osmt, IN FILE * fh) */ num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_write_all_path_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %zu records\n", num_recs); result = fprintf(fh, "#\n" "# Path Records\n" "#\n"); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_path_recs: ERR 0026: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0026: " "Write failed\n"); status = IB_ERROR; goto Exit; @@ -2406,8 +2300,7 @@ osmtest_write_all_node_recs(IN osmtest_t * const p_osmt, IN FILE * fh) result = fprintf(fh, "#\n" "# Node Records\n" "#\n"); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_node_recs: ERR 0027: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0027: " "Write failed\n"); status = IB_ERROR; goto Exit; @@ -2426,14 +2319,12 @@ osmtest_write_all_node_recs(IN osmtest_t * const p_osmt, IN FILE * fh) &context); if (status != IB_SUCCESS) { if (status != IB_SA_MAD_STATUS_NO_RECORDS) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_write_all_node_recs: ERR 0028: " + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "ERR 0028: " "failed to get node info for LID:0x%02X (%s)\n", cl_ntoh16(lid), ib_get_err_str(status)); goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_write_all_node_recs: WRN 0121: " + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "WRN 0121: " "failed to get node info for LID:0x%02X (%s)\n", cl_ntoh16(lid), ib_get_err_str(status)); status = IB_SUCCESS; @@ -2508,8 +2399,7 @@ osmtest_write_all_port_recs(IN osmtest_t * const p_osmt, IN FILE * fh) /* print header */ result = fprintf(fh, "#\n" "# PortInfo Records\n" "#\n"); if (result < 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_port_recs: ERR 0029: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0029: " "Write failed\n"); status = IB_ERROR; goto Exit; @@ -2539,15 +2429,15 @@ osmtest_write_all_port_recs(IN osmtest_t * const p_osmt, IN FILE * fh) &context); if (status != IB_SUCCESS) { if (status != IB_SA_MAD_STATUS_NO_RECORDS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_port_recs: WRN 0122: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "WRN 0122: " "Error encountered getting port info for LID:0x%04X Num:0x%02X (%s)\n", p_node_rec->lid, port_num, ib_get_err_str(status)); goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_write_all_port_recs: WRN 0123: " + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "WRN 0123: " "failed to get port info for LID:0x%04X Num:0x%02X (%s)\n", p_node_rec->lid, port_num, ib_get_err_str(status)); @@ -2647,8 +2537,7 @@ osmtest_write_all_path_recs(IN osmtest_t * const p_osmt, IN FILE * fh) &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_write_all_path_recs: ERR 012D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 012D: " "failed to get path info from LID:0x%X To LID:0x%X (%s)\n", p_src_node->rec.lid, p_dst_node->rec.lid, @@ -2717,8 +2606,7 @@ osmtest_create_inventory_file(IN osmtest_t * const p_osmt) fh = fopen(p_osmt->opt.file_name, "w"); if (fh == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_create_inventory_file: ERR 0079: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0079: " "Unable to open inventory file (%s)\n", p_osmt->opt.file_name); status = IB_ERROR; @@ -3042,8 +2930,7 @@ static ib_api_status_t osmtest_check_missing_nodes(IN osmtest_t * const p_osmt) /* * This node was not reported by the SA */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_check_missing_nodes: ERR 0080: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0080: " "Missing node 0x%016" PRIx64 "\n", cl_ntoh64(p_node->rec.node_info.node_guid)); status = IB_ERROR; @@ -3075,8 +2962,7 @@ static ib_api_status_t osmtest_check_missing_ports(IN osmtest_t * const p_osmt) /* * This port was not reported by the SA */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_check_missing_ports: ERR 0081: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0081: " "Missing port LID:0x%X Num:0x%X\n", cl_ntoh16(p_port->rec.lid), p_port->rec.port_num); @@ -3109,8 +2995,7 @@ static ib_api_status_t osmtest_check_missing_paths(IN osmtest_t * const p_osmt) /* * This path was not reported by the SA */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_check_missing_paths: ERR 0051: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0051: " "SA did not return path SLID 0x%X to DLID 0x%X\n", cl_ntoh16(p_path->rec.slid), cl_ntoh16(p_path->rec.dlid)); @@ -3140,8 +3025,7 @@ osmtest_path_rec_kay_is_valid(IN osmtest_t * const p_osmt, IN const path_t * const p_path) { if ((p_path->comp.dlid == 0) || (p_path->comp.slid == 0)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_path_rec_kay_is_valid: ERR 0168: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0168: " "SLID and DLID must be specified for defined paths\n"); return (FALSE); } @@ -3161,12 +3045,9 @@ osmtest_validate_path_data(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_data: " - "Checking path SLID 0x%X to DLID 0x%X\n", - cl_ntoh16(p_rec->slid), cl_ntoh16(p_rec->dlid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "Checking path SLID 0x%X to DLID 0x%X\n", + cl_ntoh16(p_rec->slid), cl_ntoh16(p_rec->dlid)); status = osmtest_get_local_port_lmc(p_osmt, p_osmt->local_port.lid, &lmc); @@ -3181,8 +3062,7 @@ osmtest_validate_path_data(IN osmtest_t * const p_osmt, * Has this record already been returned? */ if (p_path->count != 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_data: ERR 0056: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0056: " "Already received path SLID 0x%X to DLID 0x%X\n", cl_ntoh16(p_rec->slid), cl_ntoh16(p_rec->dlid)); status = IB_ERROR; @@ -3191,8 +3071,7 @@ osmtest_validate_path_data(IN osmtest_t * const p_osmt, } else { /* Also, this doesn't detect fewer than the correct number of paths being returned */ if (p_path->count >= (uint32_t) (1 << (2 * lmc))) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_data: ERR 0052: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0052: " "Already received path SLID 0x%X to DLID 0x%X count %d LMC %d\n", cl_ntoh16(p_rec->slid), cl_ntoh16(p_rec->dlid), p_path->count, lmc); @@ -3210,8 +3089,7 @@ osmtest_validate_path_data(IN osmtest_t * const p_osmt, p_path->rec.dgid.unicast.interface_id) != (p_path->comp.dgid.unicast.interface_id & p_rec->dgid.unicast.interface_id)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_data: ERR 0169: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0169: " "DGID mismatch on path SLID 0x%X to DLID 0x%X\n" "\t\t\t\tExpected 0x%016" PRIx64 " 0x%016" PRIx64 "\n" "\t\t\t\tReceived 0x%016" PRIx64 " 0x%016" PRIx64 "\n", @@ -3232,8 +3110,7 @@ osmtest_validate_path_data(IN osmtest_t * const p_osmt, p_path->rec.sgid.unicast.interface_id) != (p_path->comp.sgid.unicast.interface_id & p_rec->sgid.unicast.interface_id)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_data: ERR 0057: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0057: " "SGID mismatch on path SLID 0x%X to DLID 0x%X\n" "\t\t\t\tExpected 0x%016" PRIx64 " 0x%016" PRIx64 ",\n" "\t\t\t\tReceived 0x%016" PRIx64 " 0x%016" PRIx64 ".\n", @@ -3252,8 +3129,7 @@ osmtest_validate_path_data(IN osmtest_t * const p_osmt, */ if ((p_path->comp.pkey & p_path->rec.pkey) != (p_path->comp.pkey & p_rec->pkey)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_data: ERR 0012: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0012: " "PKEY mismatch on path SLID 0x%X to DLID 0x%X\n" "\t\t\t\tExpected 0x%X, received 0x%X\n", cl_ntoh16(p_path->rec.slid), @@ -3279,20 +3155,15 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: " - "Checking node 0x%016" PRIx64 ", LID 0x%X\n", - cl_ntoh64(p_rec->node_info.node_guid), - cl_ntoh16(p_rec->lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "Checking node 0x%016" PRIx64 ", LID 0x%X\n", + cl_ntoh64(p_rec->node_info.node_guid), cl_ntoh16(p_rec->lid)); /* * Has this record already been returned? */ if (p_node->count != 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0013: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0013: " "Already received node 0x%016" PRIx64 "\n", cl_ntoh64(p_node->rec.node_info.node_guid)); status = IB_ERROR; @@ -3306,8 +3177,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, */ if ((p_node->comp.lid & p_node->rec.lid) != (p_node->comp.lid & p_rec->lid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0014: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0014: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected LID 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), @@ -3320,8 +3190,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, p_node->rec.node_info.base_version) != (p_node->comp.node_info.base_version & p_rec->node_info.base_version)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0015: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0015: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected base_version 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), @@ -3336,8 +3205,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, p_node->rec.node_info.class_version) != (p_node->comp.node_info.class_version & p_rec->node_info.class_version)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0016: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0016: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected class_version 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), @@ -3351,8 +3219,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, if ((p_node->comp.node_info.node_type & p_node->rec.node_info.node_type) != (p_node->comp.node_info.node_type & p_rec->node_info.node_type)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0017: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0017: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected node_type 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), @@ -3366,8 +3233,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, if ((p_node->comp.node_info.sys_guid & p_node->rec.node_info.sys_guid) != (p_node->comp.node_info.sys_guid & p_rec->node_info.sys_guid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0018: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0018: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected sys_guid 0x%016" PRIx64 ", received 0x%016" PRIx64 "\n", @@ -3382,8 +3248,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, if ((p_node->comp.node_info.node_guid & p_node->rec.node_info.node_guid) != (p_node->comp.node_info.node_guid & p_rec->node_info.node_guid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0019: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0019: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected node_guid 0x%016" PRIx64 ", received 0x%016" PRIx64 "\n", @@ -3398,8 +3263,7 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, if ((p_node->comp.node_info.port_guid & p_node->rec.node_info.port_guid) != (p_node->comp.node_info.port_guid & p_rec->node_info.port_guid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0031: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0031: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" "\t\t\t\tExpected port_guid 0x%016" PRIx64 ", received 0x%016" PRIx64 "\n", @@ -3415,11 +3279,9 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, p_node->rec.node_info.partition_cap) != (p_node->comp.node_info.partition_cap & p_rec->node_info.partition_cap)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0032: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0032: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" - "\t\t\t\tExpected partition_cap 0x%X" - ", received 0x%X\n", + "\t\t\t\tExpected partition_cap 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), cl_ntoh16(p_rec->lid), cl_ntoh16(p_node->rec.node_info.partition_cap), @@ -3431,11 +3293,9 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, if ((p_node->comp.node_info.device_id & p_node->rec.node_info.device_id) != (p_node->comp.node_info.device_id & p_rec->node_info.device_id)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0033: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0033: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" - "\t\t\t\tExpected device_id 0x%X" - ", received 0x%X\n", + "\t\t\t\tExpected device_id 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), cl_ntoh16(p_rec->lid), cl_ntoh16(p_node->rec.node_info.device_id), @@ -3447,11 +3307,9 @@ osmtest_validate_node_data(IN osmtest_t * const p_osmt, if ((p_node->comp.node_info.revision & p_node->rec.node_info.revision) != (p_node->comp.node_info.revision & p_rec->node_info.revision)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_data: ERR 0034: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0034: " "Field mismatch node 0x%016" PRIx64 ", LID 0x%X\n" - "\t\t\t\tExpected revision 0x%X" - ", received 0x%X\n", + "\t\t\t\tExpected revision 0x%X, received 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), cl_ntoh16(p_rec->lid), cl_ntoh32(p_node->rec.node_info.revision), @@ -3483,8 +3341,7 @@ osmtest_validate_node_rec(IN osmtest_t * const p_osmt, p_tbl = &p_osmt->exp_subn.node_lid_tbl; p_node = (node_t *) cl_qmap_get(p_tbl, p_rec->lid); if (p_node == (node_t *) cl_qmap_end(p_tbl)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_node_rec: ERR 0035: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0035: " "Unexpected node 0x%016" PRIx64 ", LID 0x%X\n", cl_ntoh64(p_rec->node_info.node_guid), cl_ntoh16(p_rec->lid)); @@ -3510,19 +3367,15 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: " - "Checking port LID 0x%X, Num 0x%X\n", - cl_ntoh16(p_rec->lid), p_rec->port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "Checking port LID 0x%X, Num 0x%X\n", + cl_ntoh16(p_rec->lid), p_rec->port_num); /* * Has this record already been returned? */ if (p_port->count != 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0036: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0036: " "Already received port LID 0x%X, Num 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num); status = IB_ERROR; @@ -3536,8 +3389,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, */ if ((p_port->comp.lid & p_port->rec.lid) != (p_port->comp.lid & p_rec->lid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0037: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0037: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected LID 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3548,8 +3400,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_num & p_port->rec.port_num) != (p_port->comp.port_num & p_rec->port_num)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0038: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0038: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected port_num 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3560,8 +3411,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.m_key & p_port->rec.port_info.m_key) != (p_port->comp.port_info.m_key & p_rec->port_info.m_key)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0039: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0039: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected m_key 0x%016" PRIx64 ", received 0x%016" PRIx64 "\n", cl_ntoh16(p_rec->lid), @@ -3575,8 +3425,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, subnet_prefix) != (p_port->comp.port_info.subnet_prefix & p_rec->port_info. subnet_prefix)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0040: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0040: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected subnet_prefix 0x%016" PRIx64 ", received 0x%016" PRIx64 "\n", cl_ntoh16(p_rec->lid), @@ -3589,8 +3438,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.base_lid & p_port->rec.port_info. base_lid) != (p_port->comp.port_info.base_lid & p_rec->port_info.base_lid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0041: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0041: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected base_lid 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3604,8 +3452,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, master_sm_base_lid) != (p_port->comp.port_info.master_sm_base_lid & p_rec->port_info. master_sm_base_lid)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0042: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0042: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected master_sm_base_lid 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3619,8 +3466,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, capability_mask) != (p_port->comp.port_info.capability_mask & p_rec->port_info. capability_mask)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0043: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0043: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected capability_mask 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3633,8 +3479,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.diag_code & p_port->rec.port_info. diag_code) != (p_port->comp.port_info.diag_code & p_rec->port_info.diag_code)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0044: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0044: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected diag_code 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3648,8 +3493,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, m_key_lease_period) != (p_port->comp.port_info.m_key_lease_period & p_rec->port_info. m_key_lease_period)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0045: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0045: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected m_key_lease_period 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3663,8 +3507,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, local_port_num) != (p_port->comp.port_info.local_port_num & p_rec->port_info. local_port_num)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0046: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0046: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected local_port_num 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3678,8 +3521,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, link_width_enabled) != (p_port->comp.port_info.link_width_enabled & p_rec->port_info. link_width_enabled)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0047: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0047: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected link_width_enabled 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3693,8 +3535,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, port_info.link_width_supported) != (p_port->comp.port_info.link_width_supported & p_rec->port_info. link_width_supported)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0048: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0048: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected link_width_supported 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3708,8 +3549,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, link_width_active) != (p_port->comp.port_info.link_width_active & p_rec->port_info. link_width_active)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0049: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0049: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected link_width_active 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3722,8 +3562,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.link_speed & p_port->rec.port_info. link_speed) != (p_port->comp.port_info.link_speed & p_rec->port_info.link_speed)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0054: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0054: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected link_speed 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3737,8 +3576,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, state_info1) != (p_port->comp.port_info.state_info1 & p_rec->port_info. state_info1)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0055: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0055: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected state_info1 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3752,8 +3590,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, state_info2) != (p_port->comp.port_info.state_info2 & p_rec->port_info. state_info2)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0058: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0058: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected state_info2 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3766,8 +3603,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.mkey_lmc & p_port->rec.port_info. mkey_lmc) != (p_port->comp.port_info.mkey_lmc & p_rec->port_info.mkey_lmc)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0059: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0059: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected mkey_lmc 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3780,8 +3616,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.link_speed & p_port->rec.port_info. link_speed) != (p_port->comp.port_info.link_speed & p_rec->port_info.link_speed)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0060: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0060: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected link_speed 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3794,8 +3629,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.mtu_smsl & p_port->rec.port_info. mtu_smsl) != (p_port->comp.port_info.mtu_smsl & p_rec->port_info.mtu_smsl)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0061: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0061: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected mtu_smsl 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3807,8 +3641,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.vl_cap & p_port->rec.port_info.vl_cap) != (p_port->comp.port_info.vl_cap & p_rec->port_info.vl_cap)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0062: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0062: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected vl_cap 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3821,8 +3654,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, vl_high_limit) != (p_port->comp.port_info.vl_high_limit & p_rec->port_info. vl_high_limit)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0082: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0082: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected vl_high_limit 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3836,8 +3668,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, vl_arb_high_cap) != (p_port->comp.port_info.vl_arb_high_cap & p_rec->port_info. vl_arb_high_cap)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0083: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0083: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected vl_arb_high_cap 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3851,8 +3682,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, vl_arb_low_cap) != (p_port->comp.port_info.vl_arb_low_cap & p_rec->port_info. vl_arb_low_cap)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0084: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0084: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected vl_arb_low_cap 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3864,8 +3694,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.mtu_cap & p_port->rec.port_info.mtu_cap) != (p_port->comp.port_info.mtu_cap & p_rec->port_info.mtu_cap)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0085: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0085: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected mtu_cap 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3880,8 +3709,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, vl_stall_life) != (p_port->comp.port_info.vl_stall_life & p_rec->port_info. vl_stall_life)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 012F: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 012F: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected vl_stall_life 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3895,8 +3723,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.vl_enforce & p_port->rec.port_info. vl_enforce) != (p_port->comp.port_info.vl_enforce & p_rec->port_info.vl_enforce)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0086: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0086: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected vl_enforce 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3910,8 +3737,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, m_key_violations) != (p_port->comp.port_info.m_key_violations & p_rec->port_info. m_key_violations)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0087: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0087: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected m_key_violations 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3925,8 +3751,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, p_key_violations) != (p_port->comp.port_info.p_key_violations & p_rec->port_info. p_key_violations)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0088: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0088: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected p_key_violations 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3940,8 +3765,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, q_key_violations) != (p_port->comp.port_info.q_key_violations & p_rec->port_info. q_key_violations)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0089: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0089: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected q_key_violations 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3954,8 +3778,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, if ((p_port->comp.port_info.guid_cap & p_port->rec.port_info. guid_cap) != (p_port->comp.port_info.guid_cap & p_rec->port_info.guid_cap)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0090: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0090: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected guid_cap 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3969,8 +3792,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, subnet_timeout) != (p_port->comp.port_info.subnet_timeout & p_rec->port_info. subnet_timeout)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0091: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0091: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected subnet_timeout 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3984,8 +3806,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, resp_time_value) != (p_port->comp.port_info.resp_time_value & p_rec->port_info. resp_time_value)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0092: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0092: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected resp_time_value 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -3999,8 +3820,7 @@ osmtest_validate_port_data(IN osmtest_t * const p_osmt, error_threshold) != (p_port->comp.port_info.error_threshold & p_rec->port_info. error_threshold)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_data: ERR 0093: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0093: " "Field mismatch port LID 0x%X Num:0x%X\n" "\t\t\t\tExpected error_threshold 0x%X, received 0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num, @@ -4036,8 +3856,7 @@ osmtest_validate_port_rec(IN osmtest_t * const p_osmt, (port_t *) cl_qmap_get(p_tbl, port_gen_id(p_rec->lid, p_rec->port_num)); if (p_port == (port_t *) cl_qmap_end(p_tbl)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_port_rec: ERR 0094: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0094: " "Unexpected port LID 0x%X, Num:0x%X\n", cl_ntoh16(p_rec->lid), p_rec->port_num); status = IB_ERROR; @@ -4069,8 +3888,7 @@ osmtest_validate_path_rec(IN osmtest_t * const p_osmt, p_tbl = &p_osmt->exp_subn.path_tbl; p_path = (path_t *) cl_qmap_get(p_tbl, osmtest_path_rec_key_get(p_rec)); if (p_path == (path_t *) cl_qmap_end(p_tbl)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_path_rec: ERR 0095: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0095: " "Unexpected path SLID 0x%X to DLID 0x%X\n", cl_ntoh16(p_rec->slid), cl_ntoh16(p_rec->dlid)); status = IB_ERROR; @@ -4109,8 +3927,7 @@ osmtest_validate_all_node_recs(IN osmtest_t * const p_osmt) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_node_recs: ERR 0096: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0096: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4118,11 +3935,8 @@ osmtest_validate_all_node_recs(IN osmtest_t * const p_osmt) num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_all_node_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %zu records\n", + num_recs); /* * Compare the received records to the database. @@ -4135,8 +3949,7 @@ osmtest_validate_all_node_recs(IN osmtest_t * const p_osmt) status = osmtest_validate_node_rec(p_osmt, p_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_node_recs: ERR 0097: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0097: " "osmtest_valid_node_rec failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4147,8 +3960,7 @@ osmtest_validate_all_node_recs(IN osmtest_t * const p_osmt) status = osmtest_check_missing_nodes(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_node_recs: ERR 0098: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0098: " "osmtest_check_missing_nodes failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4189,8 +4001,7 @@ osmtest_validate_all_guidinfo_recs(IN osmtest_t * const p_osmt) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_guidinfo_recs: ERR 0099: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0099: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4198,11 +4009,8 @@ osmtest_validate_all_guidinfo_recs(IN osmtest_t * const p_osmt) num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_all_guidinfo_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %zu records\n", + num_recs); /* No validation as yet */ @@ -4242,8 +4050,7 @@ osmtest_validate_all_path_recs(IN osmtest_t * const p_osmt) sizeof(*p_rec), &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_path_recs: ERR 009A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 009A: " "osmtest_get_all_recs failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4251,11 +4058,8 @@ osmtest_validate_all_path_recs(IN osmtest_t * const p_osmt) num_recs = context.result.result_cnt; - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_all_path_recs: " - "Received %zu records\n", num_recs); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %zu records\n", + num_recs); /* * Compare the received records to the database. @@ -4268,8 +4072,7 @@ osmtest_validate_all_path_recs(IN osmtest_t * const p_osmt) status = osmtest_validate_path_rec(p_osmt, p_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_path_recs: ERR 0100: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0100: " "osmtest_validate_path_rec failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4278,8 +4081,7 @@ osmtest_validate_all_path_recs(IN osmtest_t * const p_osmt) status = osmtest_check_missing_paths(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_all_path_recs: ERR 0101: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0101: " "osmtest_check_missing_paths failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -4316,12 +4118,9 @@ osmtest_get_link_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_link_rec_by_lid: " - "Getting link record from LID 0x%02X to LID 0x%02X\n", - cl_ntoh16(from_lid), cl_ntoh16(to_lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting link record from LID 0x%02X to LID 0x%02X\n", + cl_ntoh16(from_lid), cl_ntoh16(to_lid)); /* * Do a blocking query for this record in the subnet. @@ -4356,8 +4155,7 @@ osmtest_get_link_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_link_rec_by_lid: ERR 007A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 007A: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4365,14 +4163,13 @@ osmtest_get_link_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_link_rec_by_lid: ERR 007B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 007B: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "osmtest_get_link_rec_by_lid: " "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4404,12 +4201,8 @@ osmtest_get_guidinfo_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_guidinfo_rec_by_lid: " - "Getting GUIDInfo record for LID 0x%02X\n", - cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting GUIDInfo record for LID 0x%02X\n", cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -4441,8 +4234,7 @@ osmtest_get_guidinfo_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_guidinfo_rec_by_lid: ERR 007C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 007C: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4450,15 +4242,13 @@ osmtest_get_guidinfo_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_guidinfo_rec_by_lid: ERR 007D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 007D: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_guidinfo_rec_by_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4490,12 +4280,8 @@ osmtest_get_pkeytbl_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_pkeytbl_rec_by_lid: " - "Getting PKeyTable record for LID 0x%02X\n", - cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting PKeyTable record for LID 0x%02X\n", cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -4527,8 +4313,7 @@ osmtest_get_pkeytbl_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_pkeytbl_rec_by_lid: ERR 007E: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 007E: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4536,15 +4321,13 @@ osmtest_get_pkeytbl_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_pkeytbl_rec_by_lid: ERR 007F: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 007F: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_pkeytbl_rec_by_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4575,12 +4358,8 @@ osmtest_get_sw_info_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_sw_info_rec_by_lid: " - "Getting SwitchInfo record for LID 0x%02X\n", - cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting SwitchInfo record for LID 0x%02X\n", cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -4613,8 +4392,7 @@ osmtest_get_sw_info_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_sw_info_rec_by_lid: ERR 006C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 006C: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4622,15 +4400,13 @@ osmtest_get_sw_info_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_sw_info_rec_by_lid: ERR 006D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 006D: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_sw_info_rec_by_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4661,11 +4437,8 @@ osmtest_get_lft_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_lft_rec_by_lid: " - "Getting LFT record for LID 0x%02X\n", cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting LFT record for LID 0x%02X\n", cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -4698,8 +4471,7 @@ osmtest_get_lft_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_lft_rec_by_lid: ERR 008A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 008A: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4707,15 +4479,13 @@ osmtest_get_lft_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_lft_rec_by_lid: ERR 008B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 008B: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_lft_rec_by_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4746,11 +4516,8 @@ osmtest_get_mft_rec_by_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_get_mft_rec_by_lid: " - "Getting MFT record for LID 0x%02X\n", cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Getting MFT record for LID 0x%02X\n", cl_ntoh16(lid)); /* * Do a blocking query for this record in the subnet. @@ -4783,8 +4550,7 @@ osmtest_get_mft_rec_by_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_mft_rec_by_lid: ERR 009B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 009B: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4792,15 +4558,13 @@ osmtest_get_mft_rec_by_lid(IN osmtest_t * const p_osmt, status = p_context->result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_mft_rec_by_lid: ERR 009C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 009C: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_mft_rec_by_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4880,8 +4644,7 @@ osmtest_sminfo_record_request(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_sminfo_record_request: ERR 008C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 008C: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -4890,8 +4653,7 @@ osmtest_sminfo_record_request(IN osmtest_t * const p_osmt, if (status != IB_SUCCESS) { if (status != IB_INVALID_PARAMETER) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_sminfo_record_request: ERR 008D: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 008D: " "ib_query failed (%s)\n", ib_get_err_str(status)); } @@ -4899,8 +4661,7 @@ osmtest_sminfo_record_request(IN osmtest_t * const p_osmt, p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_sminfo_record_request: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -4996,8 +4757,7 @@ osmtest_informinfo_request(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_informinfo_request: ERR 008E: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 008E: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -5006,8 +4766,7 @@ osmtest_informinfo_request(IN osmtest_t * const p_osmt, if (status != IB_SUCCESS) { if (status != IB_INVALID_PARAMETER) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_informinfo_request: ERR 008F: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 008F: " "ib_query failed (%s)\n", ib_get_err_str(status)); } @@ -5015,8 +4774,7 @@ osmtest_informinfo_request(IN osmtest_t * const p_osmt, p_mad = osm_madw_get_mad_ptr(p_context->result. p_result_madw); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_informinfo_request: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(p_mad)); @@ -5051,16 +4809,14 @@ osmtest_validate_single_path_rec_lid_pair(IN osmtest_t * const p_osmt, p_path->rec.slid, p_path->rec.dlid, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_lid_pair: ERR 0102: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0102: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } num_recs = context.result.result_cnt; if (num_recs != 1) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_lid_pair: ERR 0103: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0103: " "Too many records. Expected 1, received %zu\n", num_recs); @@ -5071,8 +4827,7 @@ osmtest_validate_single_path_rec_lid_pair(IN osmtest_t * const p_osmt, status = osmtest_validate_path_data(p_osmt, p_path, p_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_lid_pair: ERR 0104: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0104: " "osmtest_validate_path_data failed (%s)\n", ib_get_err_str(status)); } @@ -5110,12 +4865,8 @@ osmtest_validate_single_node_rec_lid(IN osmtest_t * const p_osmt, OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_validate_single_node_rec_lid: " - "Getting NodeRecord for node with LID 0x%X\n", - cl_ntoh16(lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Getting NodeRecord for node with LID 0x%X\n", cl_ntoh16(lid)); memset(&context, 0, sizeof(context)); memset(&req, 0, sizeof(req)); @@ -5142,8 +4893,7 @@ osmtest_validate_single_node_rec_lid(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_node_rec_lid: ERR 0105: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0105: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -5151,13 +4901,11 @@ osmtest_validate_single_node_rec_lid(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_node_rec_lid: ERR 0106: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0106: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_node_rec_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -5167,8 +4915,7 @@ osmtest_validate_single_node_rec_lid(IN osmtest_t * const p_osmt, } num_recs = context.result.result_cnt; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_node_rec_lid: " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "Received %d nodes\n", num_recs); for (i = 0; i < num_recs; i++) { @@ -5177,8 +4924,7 @@ osmtest_validate_single_node_rec_lid(IN osmtest_t * const p_osmt, status = osmtest_validate_node_rec(p_osmt, p_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_node_rec_lid: ERR 0107: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0107: " "osmtest_validate_node_data failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5219,8 +4965,7 @@ osmtest_validate_single_port_rec_lid(IN osmtest_t * const p_osmt, p_port->rec.lid, p_port->rec.port_num, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_port_rec_lid: ERR 0108: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0108: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5230,8 +4975,7 @@ osmtest_validate_single_port_rec_lid(IN osmtest_t * const p_osmt, p_rec = osmv_get_query_portinfo_rec(context.result.p_result_madw, 0); status = osmtest_validate_port_rec(p_osmt, p_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_port_rec_lid: ERR 0109: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0109: " "osmtest_validate_port_data failed (%s)\n", ib_get_err_str(status)); } @@ -5270,14 +5014,10 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, memset(&req, 0, sizeof(req)); memset(&context, 0, sizeof(context)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: " - "\n\t\t\t\tChecking src 0x%016" PRIx64 - " to dest 0x%016" PRIx64 "\n", - cl_ntoh64(p_pair->src_guid), - cl_ntoh64(p_pair->dest_guid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "\n\t\t\t\tChecking src 0x%016" PRIx64 + " to dest 0x%016" PRIx64 "\n", + cl_ntoh64(p_pair->src_guid), cl_ntoh64(p_pair->dest_guid)); context.p_osmt = p_osmt; @@ -5293,8 +5033,7 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, status = osmv_query_sa(p_osmt->h_bind, &req); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: ERR 0110: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0110: " "ib_query failed (%s)\n", ib_get_err_str(status)); goto Exit; } @@ -5302,13 +5041,11 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, status = context.result.status; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: ERR 0111: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0111: " "ib_query failed (%s)\n", ib_get_err_str(status)); if (status == IB_REMOTE_ERROR) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Remote error = %s\n", ib_get_mad_status_str(osm_madw_get_mad_ptr (context.result. @@ -5318,9 +5055,7 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, } num_recs = context.result.result_cnt; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_path_rec_guid_pair: %zu records\n", - num_recs); + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "%zu records\n", num_recs); for (i = 0; i < num_recs; i++) { p_rec = @@ -5330,8 +5065,7 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, * Make sure the GUID values are correct */ if (p_rec->dgid.unicast.interface_id != p_pair->dest_guid) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: ERR 0112: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0112: " "Destination GUID mismatch\n" "\t\t\t\texpected 0x%016" PRIx64 ", received 0x%016" PRIx64 "\n", @@ -5341,8 +5075,7 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, } if (p_rec->sgid.unicast.interface_id != p_pair->src_guid) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: ERR 0113: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0113: " "Source GUID mismatch\n" "\t\t\t\texpected 0x%016" PRIx64 ", received 0x%016" PRIx64 ".\n", @@ -5353,8 +5086,7 @@ osmtest_validate_single_path_rec_guid_pair(IN osmtest_t * const p_osmt, status = osmtest_validate_path_rec(p_osmt, p_rec); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_rec_guid_pair: ERR 0114: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0114: " "osmtest_validate_path_rec failed (%s)\n", ib_get_err_str(status)); got_error = TRUE; @@ -5397,11 +5129,8 @@ osmtest_validate_single_path_recs(IN osmtest_t * const p_osmt) OSM_LOG_ENTER(&p_osmt->log); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_path_recs: " - "Validating individual path record queries\n"); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Validating individual path record queries\n"); p_path_tbl = &p_osmt->exp_subn.path_tbl; osmtest_prepare_db(p_osmt); @@ -5421,17 +5150,13 @@ osmtest_validate_single_path_recs(IN osmtest_t * const p_osmt) p_path = (path_t *) cl_qmap_next(&p_path->map_item); } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_path_recs: " - "Total of %u path records validated using LID based query\n", - cnt); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Total of %u path records validated using LID based query\n", + cnt); status = osmtest_check_missing_paths(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_recs: ERR 0115: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0115: " "osmtest_check_missing_paths failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5456,17 +5181,13 @@ osmtest_validate_single_path_recs(IN osmtest_t * const p_osmt) p_path = (path_t *) cl_qmap_next(&p_path->map_item); } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_path_recs: " - "Total of %u path records validated using GUID based query\n", - cnt); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Total of %u path records validated using GUID based query\n", + cnt); status = osmtest_check_missing_paths(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_path_recs: ERR 0116: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0116: " "osmtest_check_missing_paths failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5493,11 +5214,8 @@ osmtest_validate_single_node_recs(IN osmtest_t * const p_osmt) osmtest_prepare_db(p_osmt); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_node_recs: " - "Validating individual node record queries\n"); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Validating individual node record queries\n"); /* * Walk the list of all node records, and ask for each one @@ -5509,8 +5227,7 @@ osmtest_validate_single_node_recs(IN osmtest_t * const p_osmt) (ib_net16_t) cl_qmap_key((cl_map_item_t *) p_node), p_node); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_node_recs: ERR 011A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 011A: " "osmtest_validate_single_node_rec_lid (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5519,16 +5236,12 @@ osmtest_validate_single_node_recs(IN osmtest_t * const p_osmt) p_node = (node_t *) cl_qmap_next(&p_node->map_item); } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_node_recs: " - "Total of %u node records validated\n", cnt); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Total of %u node records validated\n", cnt); status = osmtest_check_missing_nodes(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_node_recs: ERR 0117: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0117: " "osmtest_check_missing_nodes (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5555,11 +5268,8 @@ osmtest_validate_single_port_recs(IN osmtest_t * const p_osmt) osmtest_prepare_db(p_osmt); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_port_recs: " - "Validating individual port record queries\n"); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Validating individual port record queries\n"); /* * Walk the list of all port records, and ask for each one @@ -5569,8 +5279,7 @@ osmtest_validate_single_port_recs(IN osmtest_t * const p_osmt) while (p_port != (port_t *) cl_qmap_end(p_port_key_tbl)) { status = osmtest_validate_single_port_rec_lid(p_osmt, p_port); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_port_recs: ERR 011B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 011B: " "osmtest_validate_single_port_rec_lid (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5579,16 +5288,12 @@ osmtest_validate_single_port_recs(IN osmtest_t * const p_osmt) p_port = (port_t *) cl_qmap_next(&p_port->map_item); } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_VERBOSE)) { - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_validate_single_port_recs: " - "Total of %u port records validated\n", cnt); - } + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, + "Total of %u port records validated\n", cnt); status = osmtest_check_missing_ports(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_validate_single_port_recs: ERR 0118: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0118: " "osmtest_check_missing_paths failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -5681,16 +5386,13 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) memset(&context, 0, sizeof(context)); memset(&request, 0, sizeof(request)); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_multipath_rec(p_osmt, &request, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { status = IB_ERROR; @@ -5703,16 +5405,13 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) request.sgid_count = 1; ib_gid_set_default(&request.gids[0], portguid); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_multipath_rec(p_osmt, &request, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { status = IB_ERROR; @@ -5730,16 +5429,13 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) request.gids[1].unicast.prefix = CL_HTON64(0xff12401bffff0000ULL); request.gids[1].unicast.interface_id = CL_HTON64(0x00000000ffffffffULL); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_multipath_rec(p_osmt, &request, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { status = IB_ERROR; @@ -5756,16 +5452,13 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) request.gids[0].unicast.interface_id = CL_HTON64(0x00000000ffffffffULL); ib_gid_set_default(&request.gids[1], portguid); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_multipath_rec(p_osmt, &request, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { status = IB_ERROR; @@ -5837,16 +5530,13 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) memset(&context, 0, sizeof(context)); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_START "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_pkeytbl_rec_by_lid(p_osmt, test_lid, 0, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_multipath_rec: " EXPECTING_ERRORS_END "\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { status = IB_ERROR; @@ -5927,8 +5617,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) status = IB_ERROR; goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_sminfo_request: " "IS EXPECTED ERROR ^^^^\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "IS EXPECTED ERROR ^^^^\n"); } memset(&sm_info_rec_opt, 0, sizeof(sm_info_rec_opt)); @@ -5983,8 +5672,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) } /* InformInfoRecord tests */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a BAD - Set Unsubscribe request\n"); memset(&inform_info_opt, 0, sizeof(inform_info_opt)); memset(&inform_info_rec_opt, 0, sizeof(inform_info_rec_opt)); @@ -5997,13 +5685,11 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) status = IB_ERROR; goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "InformInfoRecord " "IS EXPECTED ERROR ^^^^\n"); } - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - Empty GetTable request\n"); memset(&context, 0, sizeof(context)); status = @@ -6014,8 +5700,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* InformInfo tests */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a BAD - Empty Get request " "(should fail with NO_RECORDS)\n"); memset(&context, 0, sizeof(context)); @@ -6026,13 +5711,11 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) status = IB_ERROR; goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "InformInfo " "IS EXPECTED ERROR ^^^^\n"); } - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a BAD - Set Unsubscribe request\n"); memset(&context, 0, sizeof(context)); status = osmtest_informinfo_request(p_osmt, IB_MAD_ATTR_INFORM_INFO, @@ -6042,14 +5725,12 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) status = IB_ERROR; goto Exit; } else { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_informinfo_request: InformInfo UnSubscribe " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "InformInfo UnSubscribe " "IS EXPECTED ERROR ^^^^\n"); } /* Now subscribe */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a Good - Set Subscribe request\n"); inform_info_opt.subscribe = TRUE; memset(&context, 0, sizeof(context)); @@ -6060,8 +5741,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Now unsubscribe (QPN needs to be 1 to work) */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a Good - Set Unsubscribe request\n"); inform_info_opt.subscribe = FALSE; inform_info_opt.qpn = 1; @@ -6073,8 +5753,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Now subscribe again */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a Good - Set Subscribe request\n"); inform_info_opt.subscribe = TRUE; inform_info_opt.qpn = 1; @@ -6086,8 +5765,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Subscribe over existing subscription */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a Good - Set Subscribe (again) request\n"); inform_info_opt.qpn = 0; memset(&context, 0, sizeof(context)); @@ -6099,8 +5777,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) /* More InformInfoRecord tests */ /* RID lookup (with currently invalid enum) */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - GetTable by GID\n"); ib_gid_set_default(&inform_info_rec_opt.subscriber_gid, p_osmt->local_port.port_guid); @@ -6114,8 +5791,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Enum lookup */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - GetTable (subscriber_enum == 0) request\n"); inform_info_rec_opt.subscriber_enum = 0; memset(&context, 0, sizeof(context)); @@ -6127,8 +5803,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Get all InformInfoRecords */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - GetTable (ALL records) request\n"); memset(&inform_info_rec_opt, 0, sizeof(inform_info_rec_opt)); memset(&context, 0, sizeof(context)); @@ -6140,8 +5815,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Another subscription */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending another Good - Set Subscribe (again) request\n"); inform_info_opt.qpn = 0; inform_info_opt.trap = 0x1234; @@ -6153,8 +5827,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Get all InformInfoRecords again */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - GetTable (ALL records) request\n"); memset(&inform_info_rec_opt, 0, sizeof(inform_info_rec_opt)); memset(&context, 0, sizeof(context)); @@ -6167,8 +5840,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) /* Cleanup subscriptions before further testing */ /* Does order of deletion matter ? Test this !!! */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo " "Sending a Good - Set (cleanup) request\n"); inform_info_opt.subscribe = FALSE; inform_info_opt.qpn = 1; @@ -6180,8 +5852,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Get all InformInfoRecords again */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - GetTable (ALL records) request\n"); memset(&inform_info_rec_opt, 0, sizeof(inform_info_rec_opt)); memset(&context, 0, sizeof(context)); @@ -6192,8 +5863,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) if (status != IB_SUCCESS) goto Exit; - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfo" + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfo" "Sending a Good - Set (cleanup) request\n"); inform_info_opt.subscribe = FALSE; inform_info_opt.qpn = 1; @@ -6206,8 +5876,7 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; /* Get all InformInfoRecords a final time */ - osm_log(&p_osmt->log, OSM_LOG_VERBOSE, - "osmtest_informinfo_request: InformInfoRecord " + OSM_LOG(&p_osmt->log, OSM_LOG_VERBOSE, "InformInfoRecord " "Sending a Good - GetTable (ALL records) request\n"); memset(&inform_info_rec_opt, 0, sizeof(inform_info_rec_opt)); memset(&context, 0, sizeof(context)); @@ -6294,20 +5963,17 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; memset(&context, 0, sizeof(context)); - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_path_rec_by_lid_pair(p_osmt, 0xffff, 0xffff, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { @@ -6315,21 +5981,18 @@ static ib_api_status_t osmtest_validate_against_db(IN osmtest_t * const p_osmt) goto Exit; } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_START "\n"); status = osmtest_get_path_rec_by_lid_pair(p_osmt, test_lid, 0xffff, &context); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "Got error %s\n", ib_get_err_str(status)); } - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_get_path_rec_by_lid_pair: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, EXPECTING_ERRORS_END "\n"); if (status == IB_SUCCESS) { @@ -6430,8 +6093,7 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, /* * End of file in the middle of a definition. */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_node: ERR 0119: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0119: " "Unexpected end of file\n"); status = IB_ERROR; goto Exit; @@ -6448,19 +6110,14 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, p_tok = str_get_token(&line[offset]); if (p_tok == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_node: ERR 0120: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0120: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); continue; } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "Found '%s' (line %u)\n", p_tok->str, - *p_line_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Found '%s' (line %u)\n", p_tok->str, *p_line_num); str_skip_token(line, &offset); @@ -6473,73 +6130,53 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, p_node->rec.lid = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "lid = 0x%X\n", - cl_ntoh16(p_node->rec.lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "lid = 0x%X\n", + cl_ntoh16(p_node->rec.lid)); break; case OSMTEST_TOKEN_BASE_VERSION: p_node->comp.node_info.base_version = 0xFF; p_node->rec.node_info.base_version = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "base_version = 0x%X\n", - p_node->rec.node_info.base_version); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "base_version = 0x%X\n", + p_node->rec.node_info.base_version); break; case OSMTEST_TOKEN_CLASS_VERSION: p_node->comp.node_info.class_version = 0xFF; p_node->rec.node_info.class_version = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "class_version = 0x%X\n", - p_node->rec.node_info.class_version); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "class_version = 0x%X\n", + p_node->rec.node_info.class_version); break; case OSMTEST_TOKEN_NODE_TYPE: p_node->comp.node_info.node_type = 0xFF; p_node->rec.node_info.node_type = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "node_type = 0x%X\n", - p_node->rec.node_info.node_type); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "node_type = 0x%X\n", + p_node->rec.node_info.node_type); break; case OSMTEST_TOKEN_NUM_PORTS: p_node->comp.node_info.num_ports = 0xFF; p_node->rec.node_info.num_ports = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "num_ports = 0x%X\n", - p_node->rec.node_info.num_ports); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "num_ports = 0x%X\n", + p_node->rec.node_info.num_ports); break; case OSMTEST_TOKEN_SYS_GUID: p_node->comp.node_info.sys_guid = 0xFFFFFFFFFFFFFFFFULL; p_node->rec.node_info.sys_guid = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "sys_guid = 0x%016" PRIx64 "\n", - cl_ntoh64(p_node->rec.node_info. - sys_guid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "sys_guid = 0x%016" PRIx64 "\n", + cl_ntoh64(p_node->rec.node_info.sys_guid)); break; case OSMTEST_TOKEN_NODE_GUID: @@ -6547,13 +6184,9 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, 0xFFFFFFFFFFFFFFFFULL; p_node->rec.node_info.node_guid = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "node_guid = 0x%016" PRIx64 "\n", - cl_ntoh64(p_node->rec.node_info. - node_guid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "node_guid = 0x%016" PRIx64 "\n", + cl_ntoh64(p_node->rec.node_info.node_guid)); break; case OSMTEST_TOKEN_PORT_GUID: @@ -6561,13 +6194,9 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, 0xFFFFFFFFFFFFFFFFULL; p_node->rec.node_info.port_guid = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "port_guid = 0x%016" PRIx64 "\n", - cl_ntoh64(p_node->rec.node_info. - port_guid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "port_guid = 0x%016" PRIx64 "\n", + cl_ntoh64(p_node->rec.node_info.port_guid)); break; case OSMTEST_TOKEN_PARTITION_CAP: @@ -6575,13 +6204,9 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, p_node->rec.node_info.partition_cap = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "partition_cap = 0x%X\n", - cl_ntoh16(p_node->rec.node_info. - partition_cap)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "partition_cap = 0x%X\n", + cl_ntoh16(p_node->rec.node_info.partition_cap)); break; case OSMTEST_TOKEN_DEVICE_ID: @@ -6591,26 +6216,18 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, (&line [offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "device_id = 0x%X\n", - cl_ntoh16(p_node->rec.node_info. - device_id)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "device_id = 0x%X\n", + cl_ntoh16(p_node->rec.node_info.device_id)); break; case OSMTEST_TOKEN_REVISION: p_node->comp.node_info.revision = 0xFFFFFFFF; p_node->rec.node_info.revision = cl_hton32(strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "revision = 0x%X\n", - cl_ntoh32(p_node->rec.node_info. - revision)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "revision = 0x%X\n", + cl_ntoh32(p_node->rec.node_info.revision)); break; case OSMTEST_TOKEN_PORT_NUM: @@ -6618,13 +6235,10 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, IB_NODE_INFO_PORT_NUM_MASK; p_node->rec.node_info.port_num_vendor_id |= (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "local_port_num = 0x%X\n", - ib_node_info_get_local_port_num - (&p_node->rec.node_info)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "local_port_num = 0x%X\n", + ib_node_info_get_local_port_num + (&p_node->rec.node_info)); break; case OSMTEST_TOKEN_VENDOR_ID: @@ -6632,13 +6246,10 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, IB_NODE_INFO_VEND_ID_MASK; p_node->rec.node_info.port_num_vendor_id |= cl_hton32(strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_node: " - "vendor_id = 0x%X\n", - cl_ntoh32(ib_node_info_get_vendor_id - (&p_node->rec.node_info))); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "vendor_id = 0x%X\n", + cl_ntoh32(ib_node_info_get_vendor_id + (&p_node->rec.node_info))); break; case OSMTEST_TOKEN_END: @@ -6646,8 +6257,7 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, break; default: - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_node: ERR 0121: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0121: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); @@ -6660,8 +6270,7 @@ osmtest_parse_node(IN osmtest_t * const p_osmt, * add this object to the database. */ if (p_node->comp.lid == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_node: ERR 0122: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0122: " "LID must be specified for defined nodes\n"); node_delete(p_node); goto Exit; @@ -6710,8 +6319,7 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, /* * End of file in the middle of a definition. */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_port: ERR 0123: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0123: " "Unexpected end of file\n"); status = IB_ERROR; goto Exit; @@ -6728,19 +6336,14 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_tok = str_get_token(&line[offset]); if (p_tok == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_port: ERR 0124: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0124: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); continue; } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "Found '%s' (line %u)\n", p_tok->str, - *p_line_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Found '%s' (line %u)\n", p_tok->str, *p_line_num); str_skip_token(line, &offset); @@ -6753,36 +6356,25 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.lid = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "lid = 0x%X\n", - cl_ntoh16(p_port->rec.lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "lid = 0x%X\n", + cl_ntoh16(p_port->rec.lid)); break; case OSMTEST_TOKEN_PORT_NUM: p_port->comp.port_num = 0xFF; p_port->rec.port_num = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "port_num = 0x%u\n", - p_port->rec.port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "port_num = 0x%u\n", p_port->rec.port_num); break; case OSMTEST_TOKEN_MKEY: p_port->comp.port_info.m_key = 0xFFFFFFFFFFFFFFFFULL; p_port->rec.port_info.m_key = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "m_key = 0x%016" PRIx64 "\n", - cl_ntoh64(p_port->rec.port_info.m_key)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "m_key = 0x%016" PRIx64 "\n", + cl_ntoh64(p_port->rec.port_info.m_key)); break; case OSMTEST_TOKEN_SUBN_PREF: @@ -6790,13 +6382,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, 0xFFFFFFFFFFFFFFFFULL; p_port->rec.port_info.subnet_prefix = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "subnet_prefix = 0x%016" PRIx64 "\n", - cl_ntoh64(p_port->rec.port_info. - subnet_prefix)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "subnet_prefix = 0x%016" PRIx64 "\n", + cl_ntoh64(p_port->rec.port_info.subnet_prefix)); break; case OSMTEST_TOKEN_BASE_LID: @@ -6804,13 +6392,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.base_lid = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "base_lid = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - base_lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "base_lid = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.base_lid)); break; case OSMTEST_TOKEN_SM_BASE_LID: @@ -6818,13 +6402,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.master_sm_base_lid = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "master_sm_base_lid = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - master_sm_base_lid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "master_sm_base_lid = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.master_sm_base_lid)); break; case OSMTEST_TOKEN_CAP_MASK: @@ -6832,13 +6412,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.capability_mask = cl_hton32((uint32_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "capability_mask = 0x%X\n", - cl_ntoh32(p_port->rec.port_info. - capability_mask)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "capability_mask = 0x%X\n", + cl_ntoh32(p_port->rec.port_info.capability_mask)); break; case OSMTEST_TOKEN_DIAG_CODE: @@ -6846,13 +6422,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.diag_code = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "diag_code = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - diag_code)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "diag_code = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.diag_code)); break; case OSMTEST_TOKEN_MKEY_LEASE_PER: @@ -6860,64 +6432,45 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.m_key_lease_period = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "m_key_lease_period = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - m_key_lease_period)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "m_key_lease_period = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.m_key_lease_period)); break; case OSMTEST_TOKEN_LOC_PORT_NUM: p_port->comp.port_info.local_port_num = 0xFF; p_port->rec.port_info.local_port_num = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "local_port_num = 0x%u\n", - p_port->rec.port_info.local_port_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "local_port_num = 0x%u\n", + p_port->rec.port_info.local_port_num); break; case OSMTEST_TOKEN_LINK_WID_EN: p_port->comp.port_info.link_width_enabled = 0xFF; p_port->rec.port_info.link_width_enabled = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "link_width_enabled = 0x%u\n", - p_port->rec.port_info. - link_width_enabled); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "link_width_enabled = 0x%u\n", + p_port->rec.port_info.link_width_enabled); break; case OSMTEST_TOKEN_LINK_WID_SUP: p_port->comp.port_info.link_width_supported = 0xFF; p_port->rec.port_info.link_width_supported = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "link_width_supported = 0x%u\n", - p_port->rec.port_info. - link_width_supported); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "link_width_supported = 0x%u\n", + p_port->rec.port_info.link_width_supported); break; case OSMTEST_TOKEN_LINK_WID_ACT: p_port->comp.port_info.link_width_active = 0xFF; p_port->rec.port_info.link_width_active = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "link_width_active = 0x%u\n", - p_port->rec.port_info. - link_width_active); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "link_width_active = 0x%u\n", + p_port->rec.port_info.link_width_active); break; case OSMTEST_TOKEN_LINK_SPEED_SUP: @@ -6926,13 +6479,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, strtoul(&line[offset], NULL, 0), &p_port->rec.port_info); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "link_speed_supported = 0x%u\n", - ib_port_info_get_link_speed_sup - (&p_port->rec.port_info)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "link_speed_supported = 0x%u\n", + ib_port_info_get_link_speed_sup(&p_port->rec.port_info)); break; case OSMTEST_TOKEN_PORT_STATE: @@ -6941,26 +6490,18 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, ib_port_info_set_port_state(&p_port->rec.port_info, ib_get_port_state_from_str (&line[offset])); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "port_state = 0x%u\n", - ib_port_info_get_port_state(&p_port-> - rec. - port_info)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "port_state = 0x%u\n", + ib_port_info_get_port_state(&p_port->rec.port_info)); break; case OSMTEST_TOKEN_STATE_INFO2: p_port->comp.port_info.state_info2 = 0xFF; p_port->rec.port_info.state_info2 = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "state_info2 = 0x%u\n", - p_port->rec.port_info.state_info2); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "state_info2 = 0x%u\n", + p_port->rec.port_info.state_info2); break; case OSMTEST_TOKEN_MKEY_PROT_BITS: @@ -6968,13 +6509,8 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, ib_port_info_set_mpb(&p_port->rec.port_info, (uint8_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "mpb = 0x%u\n", - ib_port_info_get_mpb(&p_port->rec. - port_info)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "mpb = 0x%u\n", + ib_port_info_get_mpb(&p_port->rec.port_info)); break; case OSMTEST_TOKEN_LMC: @@ -6982,122 +6518,87 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, ib_port_info_set_lmc(&p_port->rec.port_info, (uint8_t) strtoul(&line[offset], NULL, 0)); - - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "lmc = 0x%u\n", - ib_port_info_get_lmc(&p_port->rec. - port_info)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "lmc = 0x%u\n", + ib_port_info_get_lmc(&p_port->rec.port_info)); break; case OSMTEST_TOKEN_LINK_SPEED: p_port->comp.port_info.link_speed = 0xFF; p_port->rec.port_info.link_speed = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "link_speed = 0x%u\n", - p_port->rec.port_info.link_speed); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "link_speed = 0x%u\n", + p_port->rec.port_info.link_speed); break; case OSMTEST_TOKEN_MTU_SMSL: p_port->comp.port_info.mtu_smsl = 0xFF; p_port->rec.port_info.mtu_smsl = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "mtu_smsl = 0x%u\n", - p_port->rec.port_info.mtu_smsl); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "mtu_smsl = 0x%u\n", + p_port->rec.port_info.mtu_smsl); break; case OSMTEST_TOKEN_VL_CAP: p_port->comp.port_info.vl_cap = 0xFF; p_port->rec.port_info.vl_cap = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "vl_cap = 0x%u\n", - p_port->rec.port_info.vl_cap); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "vl_cap = 0x%u\n", + p_port->rec.port_info.vl_cap); break; case OSMTEST_TOKEN_VL_HIGH_LIMIT: p_port->comp.port_info.vl_high_limit = 0xFF; p_port->rec.port_info.vl_high_limit = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "vl_high_limit = 0x%u\n", - p_port->rec.port_info.vl_high_limit); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "vl_high_limit = 0x%u\n", + p_port->rec.port_info.vl_high_limit); break; case OSMTEST_TOKEN_VL_ARB_HIGH_CAP: p_port->comp.port_info.vl_arb_high_cap = 0xFF; p_port->rec.port_info.vl_arb_high_cap = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "vl_arb_high_cap = 0x%u\n", - p_port->rec.port_info.vl_arb_high_cap); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "vl_arb_high_cap = 0x%u\n", + p_port->rec.port_info.vl_arb_high_cap); break; case OSMTEST_TOKEN_VL_ARB_LOW_CAP: p_port->comp.port_info.vl_arb_low_cap = 0xFF; p_port->rec.port_info.vl_arb_low_cap = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "vl_arb_low_cap = 0x%u\n", - p_port->rec.port_info.vl_arb_low_cap); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "vl_arb_low_cap = 0x%u\n", + p_port->rec.port_info.vl_arb_low_cap); break; case OSMTEST_TOKEN_MTU_CAP: p_port->comp.port_info.mtu_cap = 0xFF; p_port->rec.port_info.mtu_cap = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "mtu_cap = 0x%u\n", - p_port->rec.port_info.mtu_cap); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "mtu_cap = 0x%u\n", + p_port->rec.port_info.mtu_cap); break; case OSMTEST_TOKEN_VL_STALL_LIFE: p_port->comp.port_info.vl_stall_life = 0xFF; p_port->rec.port_info.vl_stall_life = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "vl_stall_life = 0x%u\n", - p_port->rec.port_info.vl_stall_life); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "vl_stall_life = 0x%u\n", + p_port->rec.port_info.vl_stall_life); break; case OSMTEST_TOKEN_VL_ENFORCE: p_port->comp.port_info.vl_enforce = 0xFF; p_port->rec.port_info.vl_enforce = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "vl_enforce = 0x%u\n", - p_port->rec.port_info.vl_enforce); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "vl_enforce = 0x%u\n", + p_port->rec.port_info.vl_enforce); break; case OSMTEST_TOKEN_MKEY_VIOL: @@ -7105,13 +6606,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.m_key_violations = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "m_key_violations = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - m_key_violations)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "m_key_violations = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.m_key_violations)); break; case OSMTEST_TOKEN_PKEY_VIOL: @@ -7119,13 +6616,9 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.p_key_violations = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "p_key_violations = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - p_key_violations)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "p_key_violations = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.p_key_violations)); break; case OSMTEST_TOKEN_QKEY_VIOL: @@ -7133,62 +6626,45 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, p_port->rec.port_info.q_key_violations = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "q_key_violations = 0x%X\n", - cl_ntoh16(p_port->rec.port_info. - q_key_violations)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "q_key_violations = 0x%X\n", + cl_ntoh16(p_port->rec.port_info.q_key_violations)); break; case OSMTEST_TOKEN_GUID_CAP: p_port->comp.port_info.guid_cap = 0xFF; p_port->rec.port_info.guid_cap = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "guid_cap = 0x%u\n", - p_port->rec.port_info.guid_cap); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "guid_cap = 0x%u\n", + p_port->rec.port_info.guid_cap); break; case OSMTEST_TOKEN_SUBN_TIMEOUT: p_port->comp.port_info.subnet_timeout = 0x1F; p_port->rec.port_info.subnet_timeout = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "subnet_timeout = 0x%u\n", - ib_port_info_get_timeout(&p_port->rec. - port_info)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "subnet_timeout = 0x%u\n", + ib_port_info_get_timeout(&p_port->rec.port_info)); break; case OSMTEST_TOKEN_RESP_TIME_VAL: p_port->comp.port_info.resp_time_value = 0xFF; p_port->rec.port_info.resp_time_value = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "resp_time_value = 0x%u\n", - p_port->rec.port_info.resp_time_value); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "resp_time_value = 0x%u\n", + p_port->rec.port_info.resp_time_value); break; case OSMTEST_TOKEN_ERR_THRESHOLD: p_port->comp.port_info.error_threshold = 0xFF; p_port->rec.port_info.error_threshold = (uint8_t) strtoul(&line[offset], NULL, 0); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_port: " - "error_threshold = 0x%u\n", - p_port->rec.port_info.error_threshold); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "error_threshold = 0x%u\n", + p_port->rec.port_info.error_threshold); break; case OSMTEST_TOKEN_END: @@ -7196,8 +6672,7 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, break; default: - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_port: ERR 0125: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0125: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); break; @@ -7209,8 +6684,7 @@ osmtest_parse_port(IN osmtest_t * const p_osmt, * add this object to the database. */ if (p_port->comp.lid == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_port: ERR 0126: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0126: " "LID must be specified for defined ports\n"); port_delete(p_port); status = IB_ERROR; @@ -7253,8 +6727,7 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, /* * End of file in the middle of a definition. */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_path: ERR 0127: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0127: " "Unexpected end of file\n"); status = IB_ERROR; goto Exit; @@ -7271,20 +6744,15 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, p_tok = str_get_token(&line[offset]); if (p_tok == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_path: ERR 0128: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0128: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); got_error = TRUE; continue; } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_path: " - "Found '%s' (line %u)\n", p_tok->str, - *p_line_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Found '%s' (line %u)\n", p_tok->str, *p_line_num); str_skip_token(line, &offset); @@ -7305,16 +6773,10 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, p_path->rec.dgid.unicast.interface_id = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_path: " - "dgid = 0x%016" PRIx64 " 0x%016" PRIx64 - "\n", - cl_ntoh64(p_path->rec.dgid.unicast. - prefix), - cl_ntoh64(p_path->rec.dgid.unicast. - interface_id)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "dgid = 0x%016" PRIx64 " 0x%016" PRIx64 "\n", + cl_ntoh64(p_path->rec.dgid.unicast.prefix), + cl_ntoh64(p_path->rec.dgid.unicast.interface_id)); break; case OSMTEST_TOKEN_SGID: @@ -7330,16 +6792,10 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, p_path->rec.sgid.unicast.interface_id = cl_hton64(strtoull(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_path: " - "sgid = 0x%016" PRIx64 " 0x%016" PRIx64 - "\n", - cl_ntoh64(p_path->rec.sgid.unicast. - prefix), - cl_ntoh64(p_path->rec.sgid.unicast. - interface_id)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "sgid = 0x%016" PRIx64 " 0x%016" PRIx64 "\n", + cl_ntoh64(p_path->rec.sgid.unicast.prefix), + cl_ntoh64(p_path->rec.sgid.unicast.interface_id)); break; case OSMTEST_TOKEN_DLID: @@ -7347,12 +6803,8 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, p_path->rec.dlid = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_path: " - "dlid = 0x%X\n", - cl_ntoh16(p_path->rec.dlid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "dlid = 0x%X\n", + cl_ntoh16(p_path->rec.dlid)); break; case OSMTEST_TOKEN_SLID: @@ -7360,12 +6812,8 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, p_path->rec.slid = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_path: " - "slid = 0x%X\n", - cl_ntoh16(p_path->rec.slid)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "slid = 0x%X\n", + cl_ntoh16(p_path->rec.slid)); break; case OSMTEST_TOKEN_PKEY: @@ -7373,12 +6821,8 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, p_path->rec.pkey = cl_hton16((uint16_t) strtoul(&line[offset], NULL, 0)); - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_path: " - "pkey = 0x%X\n", - cl_ntoh16(p_path->rec.pkey)); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "pkey = 0x%X\n", + cl_ntoh16(p_path->rec.pkey)); break; case OSMTEST_TOKEN_END: @@ -7386,8 +6830,7 @@ osmtest_parse_path(IN osmtest_t * const p_osmt, break; default: - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_path: ERR 0129: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0129: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); got_error = TRUE; @@ -7441,8 +6884,7 @@ osmtest_parse_link(IN osmtest_t * const p_osmt, /* * End of file in the middle of a definition. */ - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_link: ERR 012A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 012A: " "Unexpected end of file\n"); status = IB_ERROR; goto Exit; @@ -7459,20 +6901,15 @@ osmtest_parse_link(IN osmtest_t * const p_osmt, p_tok = str_get_token(&line[offset]); if (p_tok == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_link: ERR 012B: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 012B: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); got_error = TRUE; continue; } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_parse_link: " - "Found '%s' (line %u)\n", p_tok->str, - *p_line_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Found '%s' (line %u)\n", p_tok->str, *p_line_num); str_skip_token(line, &offset); @@ -7489,8 +6926,7 @@ osmtest_parse_link(IN osmtest_t * const p_osmt, break; default: - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_parse_link: ERR 012C: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 012C: " "Ignoring line %u with unknown token: %s\n", *p_line_num, &line[offset]); got_error = TRUE; @@ -7522,8 +6958,7 @@ static ib_api_status_t osmtest_create_db(IN osmtest_t * const p_osmt) fh = fopen(p_osmt->opt.file_name, "r"); if (fh == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_create_db: ERR 0130: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0130: " "Unable to open inventory file (%s)\n", p_osmt->opt.file_name); status = IB_ERROR; @@ -7545,19 +6980,15 @@ static ib_api_status_t osmtest_create_db(IN osmtest_t * const p_osmt) p_tok = str_get_token(&line[offset]); if (p_tok == NULL) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_create_db: ERR 0131: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0131: " "Ignoring line %u: %s\n", line_num, &line[offset]); got_error = TRUE; continue; } - if (osm_log_is_active(&p_osmt->log, OSM_LOG_DEBUG)) { - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_create_db: " - "Found '%s' (line %u)\n", p_tok->str, line_num); - } + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, + "Found '%s' (line %u)\n", p_tok->str, line_num); switch (p_tok->val) { case OSMTEST_TOKEN_COMMENT: @@ -7580,8 +7011,7 @@ static ib_api_status_t osmtest_create_db(IN osmtest_t * const p_osmt) break; default: - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_create_db: ERR 0132: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0132: " "Ignoring line %u: %s\n", line_num, &line[offset]); got_error = TRUE; @@ -7592,8 +7022,7 @@ static ib_api_status_t osmtest_create_db(IN osmtest_t * const p_osmt) status = IB_ERROR; if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_create_db: ERR 0133: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0133: " "Bad status received during parsing (%s)\n", ib_get_err_str(status)); fclose(fh); @@ -7677,8 +7106,7 @@ osmtest_bind(IN osmtest_t * p_osmt, status = osm_vendor_get_all_port_attr(p_osmt->p_vendor, attr_array, &num_ports); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_bind: ERR 0134: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0134: " "Failure getting local port attributes (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7693,8 +7121,7 @@ osmtest_bind(IN osmtest_t * p_osmt, osmtest_get_user_port(p_osmt, attr_array, num_ports); if (num_ports == 0) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_bind: ERR 0135: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0135: " "No local ports. Unable to proceed\n"); goto Exit; } @@ -7706,8 +7133,7 @@ osmtest_bind(IN osmtest_t * p_osmt, } if (port_index == num_ports) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_bind: ERR 0136: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0136: " "No local port with guid 0x%016" PRIx64 "\n", cl_ntoh64(guid)); status = IB_NOT_FOUND; @@ -7722,8 +7148,7 @@ osmtest_bind(IN osmtest_t * p_osmt, sizeof(p_osmt->local_port)); /* bind to the SA */ - osm_log(&p_osmt->log, OSM_LOG_DEBUG, - "osmtest_bind: " + OSM_LOG(&p_osmt->log, OSM_LOG_DEBUG, "Using port with SM LID:0x%04X\n", p_osmt->local_port.sm_lid); p_osmt->max_lid = max_lid; @@ -7731,8 +7156,8 @@ osmtest_bind(IN osmtest_t * p_osmt, osmv_bind_sa(p_osmt->p_vendor, &p_osmt->mad_pool, guid); if (p_osmt->h_bind == OSM_BIND_INVALID_HANDLE) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_bind: ERR 0137: " "Unable to bind to SA\n"); + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0137: " + "Unable to bind to SA\n"); status = IB_ERROR; goto Exit; } @@ -7752,8 +7177,7 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) status = osmtest_validate_sa_class_port_info(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0138: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0138: " "Could not obtain SA ClassPortInfo (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7765,8 +7189,7 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) */ status = osmtest_create_inventory_file(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0139: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, "ERR 0139: " "Inventory file create failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7781,8 +7204,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) case 1: /* small response SA query stress */ status = osmtest_stress_small_rmpp(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0140: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0140: " "Small RMPP stress test failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7791,8 +7214,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) case 2: /* large response SA query stress */ status = osmtest_stress_large_rmpp(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0141: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0141: " "Large RMPP stress test failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7801,8 +7224,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) case 3: /* large response Path Record SA query stress */ status = osmtest_create_db(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0142: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0142: " "Database creation failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7810,16 +7233,16 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) status = osmtest_stress_large_rmpp_pr(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0143: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0143: " "Large RMPP stress test failed (%s)\n", ib_get_err_str(status)); goto Exit; } break; default: - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0144: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0144: " "Unknown stress test value %u\n", p_osmt->opt.stress); break; @@ -7836,8 +7259,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) */ status = osmtest_create_db(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0145: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0145: " "Database creation failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7845,8 +7268,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) status = osmtest_validate_against_db(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0146: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0146: " "SA validation database failure (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7856,8 +7279,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) if (p_osmt->opt.flow == OSMT_FLOW_ALL) { status = osmtest_wrong_sm_key_ignored(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0147: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0147: " "Try wrong SM_Key failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7872,8 +7295,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) */ status = osmt_run_service_records_flow(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0148: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0148: " "Service Flow failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7889,15 +7312,15 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) status = osmt_run_inform_info_flow(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0149: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0149: " "Inform Info Flow failed: (%s)\n", ib_get_err_str(status)); goto Exit; } #else - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmtest_run: The event forwarding flow " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "The event forwarding flow " "is not implemented yet!\n"); status = IB_SUCCESS; goto Exit; @@ -7912,8 +7335,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) */ status = osmtest_create_db(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 014A: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 014A: " "Database creation failed (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7923,8 +7346,8 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) osmt_run_slvl_and_vlarb_records_flow (p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0150: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0150: " "Failed to get SLtoVL and VL Arbitration Tables (%s)\n", ib_get_err_str(status)); goto Exit; @@ -7938,15 +7361,15 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) #ifdef OSM_VENDOR_INTF_MTL status = osmt_run_trap64_65_flow(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0151: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0151: " "Trap 64/65 Flow failed: (%s)\n", ib_get_err_str(status)); goto Exit; } #else - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmtest_run: The event forwarding flow " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, + "The event forwarding flow " "is not implemented yet!\n"); status = IB_SUCCESS; goto Exit; @@ -7960,16 +7383,15 @@ ib_api_status_t osmtest_run(IN osmtest_t * const p_osmt) */ status = osmt_run_mcast_flow(p_osmt); if (status != IB_SUCCESS) { - osm_log(&p_osmt->log, OSM_LOG_ERROR, - "osmtest_run: ERR 0152: " + OSM_LOG(&p_osmt->log, OSM_LOG_ERROR, + "ERR 0152: " "Multicast Flow failed: (%s)\n", ib_get_err_str(status)); goto Exit; } } - osm_log(&p_osmt->log, OSM_LOG_INFO, - "osmtest_run: " + OSM_LOG(&p_osmt->log, OSM_LOG_INFO, "\n\n***************** ALL TESTS PASS *****************\n\n"); } -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Sat Jul 26 19:04:53 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 05:04:53 +0300 Subject: [ofa-general] [PATCH] opensm/libvendor: convert osm_log() to OSM_LOG() macro In-Reply-To: <20080727020306.GJ5067@sashak.voltaire.com> References: <20080709223030.GY12551@sashak.voltaire.com> <20080709223132.GZ12551@sashak.voltaire.com> <20080709223534.GA12551@sashak.voltaire.com> <20080727020306.GJ5067@sashak.voltaire.com> Message-ID: <20080727020453.GL5067@sashak.voltaire.com> Convert logging to use OSM_LOG() macro. Signed-off-by: Sasha Khapyorsky --- opensm/libvendor/osm_pkt_randomizer.c | 21 ++--- opensm/libvendor/osm_vendor_ibumad.c | 151 ++++++++++++------------------- opensm/libvendor/osm_vendor_ibumad_sa.c | 93 ++++++------------- 3 files changed, 92 insertions(+), 173 deletions(-) diff --git a/opensm/libvendor/osm_pkt_randomizer.c b/opensm/libvendor/osm_pkt_randomizer.c index 5d800b4..4f80934 100644 --- a/opensm/libvendor/osm_pkt_randomizer.c +++ b/opensm/libvendor/osm_pkt_randomizer.c @@ -91,8 +91,7 @@ __osm_pkt_randomizer_is_path_in_fault_paths(IN osm_log_t * p_log, /* If found_path is TRUE then there is a full match of the path */ if (found_path == TRUE) { - osm_log(p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_is_path_in_fault_paths: " + OSM_LOG(p_log, OSM_LOG_VERBOSE, "Given path is in a fault path\n"); res = TRUE; break; @@ -179,8 +178,7 @@ __osm_pkt_randomizer_process_path(IN osm_log_t * p_log, if to update it there or not. */ if (rand_value % (p_pkt_rand->osm_pkt_unstable_link_rate) == 0) { - osm_log(p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_process_path: " + OSM_LOG(p_log, OSM_LOG_VERBOSE, "%s added to the fault_dr_paths list\n" "\t\t\t rand_value:%u, unstable_link_rate:%u \n", buf, rand_value, @@ -200,9 +198,7 @@ __osm_pkt_randomizer_process_path(IN osm_log_t * p_log, if (in_fault_paths == FALSE) { /* If in_fault_paths is FALSE - just ignore the path */ - osm_log(p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_process_path: " - "%s not in fault paths\n", buf); + OSM_LOG(p_log, OSM_LOG_VERBOSE, "%s not in fault paths\n", buf); goto Exit; } @@ -213,9 +209,7 @@ __osm_pkt_randomizer_process_path(IN osm_log_t * p_log, if (rand_value % (p_pkt_rand->osm_pkt_drop_rate) == 0) { /* drop the current packet */ res = TRUE; - osm_log(p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_process_path: " - "Dropping path:%s\n", buf); + OSM_LOG(p_log, OSM_LOG_VERBOSE, "Dropping path:%s\n", buf); } Exit: @@ -248,8 +242,7 @@ osm_pkt_randomizer_mad_drop(IN osm_log_t * p_log, if (__osm_pkt_randomizer_process_path (p_log, p_pkt_randomizer, &dr_path)) { /* the mad should be dropped o */ - osm_log(p_log, OSM_LOG_VERBOSE, - "osm_pkt_randomizer_mad_drop: " + OSM_LOG(p_log, OSM_LOG_VERBOSE, "mad TID: 0x%" PRIx64 " is being dropped\n", cl_ntoh64(p_smp->trans_id)); res = TRUE; @@ -294,9 +287,7 @@ osm_pkt_randomizer_init(IN OUT osm_pkt_randomizer_t ** pp_pkt_randomizer, else (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate = 20; - osm_log(p_log, OSM_LOG_VERBOSE, - "osm_pkt_randomizer_init: " - "Using OSM_PKT_DROP_RATE=%u \n" + OSM_LOG(p_log, OSM_LOG_VERBOSE, "Using OSM_PKT_DROP_RATE=%u \n" "\t\t\t\t OSM_PKT_NUM_UNSTABLE_LINKS=%u \n" "\t\t\t\t OSM_PKT_UNSTABLE_LINK_RATE=%u \n", (*pp_pkt_randomizer)->osm_pkt_drop_rate, diff --git a/opensm/libvendor/osm_vendor_ibumad.c b/opensm/libvendor/osm_vendor_ibumad.c index a1fc140..96fd01b 100644 --- a/opensm/libvendor/osm_vendor_ibumad.c +++ b/opensm/libvendor/osm_vendor_ibumad.c @@ -112,8 +112,7 @@ static void clear_madw(osm_vendor_t * p_vend) *) ((osm_madw_t *) m->v)->h_bind)-> p_mad_pool, m->v); pthread_mutex_unlock(&p_vend->match_tbl_mutex); - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "clear_madw: ERR 5401: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5401: " "evicting entry %p (tid was 0x%" PRIx64 ")\n", old_m, old_tid); goto Exit; @@ -192,8 +191,7 @@ put_madw(osm_vendor_t * p_vend, osm_madw_t * p_madw, ib_net64_t tid) lru->version = cl_atomic_inc((atomic32_t *) & p_vend->mtbl.last_version); pthread_mutex_unlock(&p_vend->match_tbl_mutex); - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "put_madw: ERR 5402: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5402: " "evicting entry %p (tid was 0x%" PRIx64 ")\n", old_lru, cl_ntoh64(old_tid)); } @@ -255,8 +253,7 @@ static void *umad_receiver(void *p_ptr) for (;;) { if (!umad && !(umad = umad_alloc(1, umad_size() + MAD_BLOCK_SIZE))) { - osm_log(p_ur->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5403: " + OSM_LOG(p_ur->p_log, OSM_LOG_ERROR, "ERR 5403: " "can't alloc MAD sized umad\n"); break; } @@ -265,8 +262,7 @@ static void *umad_receiver(void *p_ptr) if ((mad_agent = umad_recv(p_vend->umad_port_id, umad, &length, -1)) < 0) { if (length <= MAD_BLOCK_SIZE) { - osm_log(p_ur->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5404: " + OSM_LOG(p_ur->p_log, OSM_LOG_ERROR, "ERR 5404: " "recv error on MAD sized umad (%m)\n"); continue; } else { @@ -274,8 +270,8 @@ static void *umad_receiver(void *p_ptr) /* Need a larger buffer for RMPP */ umad = umad_alloc(1, umad_size() + length); if (!umad) { - osm_log(p_ur->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5405: " + OSM_LOG(p_ur->p_log, OSM_LOG_ERROR, + "ERR 5405: " "can't alloc umad length %d\n", length); continue; @@ -284,8 +280,8 @@ static void *umad_receiver(void *p_ptr) if ((mad_agent = umad_recv(p_vend->umad_port_id, umad, &length, -1)) < 0) { - osm_log(p_ur->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5406: " + OSM_LOG(p_ur->p_log, OSM_LOG_ERROR, + "ERR 5406: " "recv error on umad length %d (%m)\n", length); continue; @@ -295,8 +291,7 @@ static void *umad_receiver(void *p_ptr) if (mad_agent >= UMAD_CA_MAX_AGENTS || !(p_bind = p_vend->agents[mad_agent])) { - osm_log(p_ur->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5407: " + OSM_LOG(p_ur->p_log, OSM_LOG_ERROR, "ERR 5407: " "invalid mad agent %d - dropping\n", mad_agent); continue; } @@ -312,8 +307,7 @@ static void *umad_receiver(void *p_ptr) (osm_bind_handle_t) p_bind, MAX(length, MAD_BLOCK_SIZE), &osm_addr))) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5408: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5408: " "request for a new madw failed -- dropping packet\n"); continue; } @@ -330,8 +324,7 @@ static void *umad_receiver(void *p_ptr) /* if status != 0 then we are handling recv timeout on send */ if (umad_status(p_madw->vend_wrap.umad)) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5409: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5409: " "send completed with error" " (method=0x%X attr=0x%X trans_id=0x%" PRIx64 ") -- dropping\n", @@ -339,9 +332,8 @@ static void *umad_receiver(void *p_ptr) cl_ntoh64(mad->trans_id)); if (mad->mgmt_class != IB_MCLASS_SUBN_DIR) { /* LID routed */ - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5410: " - "class 0x%x LID 0x%x\n", + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, + "ERR 5410: class 0x%x LID 0x%x\n", mad->mgmt_class, cl_ntoh16(ib_mad_addr->lid)); } else { @@ -349,16 +341,16 @@ static void *umad_receiver(void *p_ptr) /* Direct routed SMP */ smp = (ib_smp_t *) mad; - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5411: DR SMP Hop Ptr: 0x%X\n", + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, + "ERR 5411: DR SMP Hop Ptr: 0x%X\n", smp->hop_ptr); osm_dump_smp_dr_path(p_vend->p_log, smp, OSM_LOG_ERROR); } if (!(p_req_madw = get_madw(p_vend, &mad->trans_id))) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5412: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, + "ERR 5412: " "Failed to obtain request madw for timed out MAD" "(method=0x%X attr=0x%X tid=0x%"PRIx64") -- dropping\n", mad->method, cl_ntoh16(mad->attr_id), @@ -382,8 +374,7 @@ static void *umad_receiver(void *p_ptr) p_req_madw = 0; if (ib_mad_is_response(mad) && !(p_req_madw = get_madw(p_vend, &mad->trans_id))) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5413: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5413: " "Failed to obtain request madw for received MAD" "(method=0x%X attr=0x%X tid=0x%"PRIx64") -- dropping\n", mad->method, cl_ntoh16((mad)->attr_id), @@ -396,13 +387,10 @@ static void *umad_receiver(void *p_ptr) (mad->mgmt_class != IB_MCLASS_SUBN_LID) && (ib_rmpp_is_flag_set((ib_rmpp_mad_t *) mad, IB_RMPP_FLAG_ACTIVE))) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5414: " - "class 0x%x method 0x%x" - " RMPP version %d type %d flags 0x%x" - " received -- dropping\n", - mad->mgmt_class, - mad->method, + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5414: " + "class 0x%x method 0x%x RMPP version %d type " + "%d flags 0x%x received -- dropping\n", + mad->mgmt_class, mad->method, ((ib_rmpp_mad_t *) mad)->rmpp_version, ((ib_rmpp_mad_t *) mad)->rmpp_type, ((ib_rmpp_mad_t *) mad)->rmpp_flags); @@ -468,14 +456,14 @@ osm_vendor_init(IN osm_vendor_t * const p_vend, * Open our instance of UMAD. */ if ((r = umad_init()) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: ERR 5415: Error opening UMAD\n"); + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, + "ERR 5415: Error opening UMAD\n"); } if ((n_cas = umad_get_cas_names(p_vend->ca_names, OSM_UMAD_MAX_CAS)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: ERR 5416: umad_get_cas_names failed\n"); + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, + "ERR 5416: umad_get_cas_names failed\n"); r = n_cas; goto Exit; } @@ -488,20 +476,17 @@ osm_vendor_init(IN osm_vendor_t * const p_vend, if (tmp > 0) p_vend->mtbl.max = tmp; else - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: Error:" + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "Error:" "OSM_UMAD_MAX_PENDING=%d is invalid", tmp); } - osm_log(p_vend->p_log, OSM_LOG_INFO, - "osm_vendor_init: %d pending umads specified\n", + OSM_LOG(p_vend->p_log, OSM_LOG_INFO, "%d pending umads specified\n", p_vend->mtbl.max); p_vend->mtbl.tbl = calloc(p_vend->mtbl.max, sizeof(*(p_vend->mtbl.tbl))); if (!p_vend->mtbl.tbl) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: Error:" + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "Error:" "failed to allocate vendor match table\n"); r = IB_INSUFFICIENT_MEMORY; goto Exit; @@ -522,16 +507,14 @@ osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, OSM_LOG_ENTER(p_log); if (!timeout) { - osm_log(p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 5433: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5433: " "transaction timeout cannot be 0\n"); goto Exit; } p_vend = malloc(sizeof(*p_vend)); if (p_vend == NULL) { - osm_log(p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 5417: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5417: " "Unable to allocate vendor object\n"); goto Exit; } @@ -583,8 +566,7 @@ osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, if (!*p_num_ports) { r = IB_INVALID_PARAMETER; - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5418: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5418: " "Ports in should be > 0\n"); goto Exit; } @@ -659,8 +641,7 @@ osm_vendor_open_port(IN osm_vendor_t * const p_vend, if ((r = umad_get_ca_portguids(p_vend->ca_names[ca], portguids, OSM_UMAD_MAX_CAS)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 5421: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5421: " "Unable to get CA %s port guids (%s)\n", p_vend->ca_names[ca], strerror(r)); goto Exit; @@ -675,8 +656,7 @@ osm_vendor_open_port(IN osm_vendor_t * const p_vend, /* * No local CA owns this guid! */ - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 5422: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5422: " "Unable to find requested CA guid 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); goto Exit; @@ -684,15 +664,13 @@ osm_vendor_open_port(IN osm_vendor_t * const p_vend, _found: /* Validate that node is an IB node type (not iWARP) */ if (umad_get_ca(name, &umad_ca) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542A: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 542A: " "umad_get_ca() failed\n"); goto Exit; } if (umad_ca.node_type < 1 || umad_ca.node_type > 3) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542D: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 542D: " "Type %d of node \'%s\' is not an IB node type\n", umad_ca.node_type, umad_ca.ca_name); fprintf(stderr, @@ -705,16 +683,14 @@ _found: /* Port found, try to open it */ if (umad_get_port(name, i, &p_vend->umad_port) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542B: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 542B: " "umad_get_port() failed\n"); goto Exit; } if ((umad_port_id = umad_open_port(p_vend->umad_port.ca_name, p_vend->umad_port.portnum)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542C: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 542C: " "umad_open_port() failed\n"); goto Exit; } @@ -723,8 +699,7 @@ _found: /* start receiver thread */ if (!(p_vend->receiver = calloc(1, sizeof(umad_receiver_t)))) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 5423: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5423: " "Unable to alloc receiver struct\n"); umad_close_port(umad_port_id); umad_release_port(&p_vend->umad_port); @@ -733,8 +708,7 @@ _found: goto Exit; } if (umad_receiver_start(p_vend) != 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 5420: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5420: " "umad_receiver_init failed\n"); umad_close_port(umad_port_id); umad_release_port(&p_vend->umad_port); @@ -807,12 +781,11 @@ osm_vendor_bind(IN osm_vendor_t * const p_vend, port_guid = p_user_bind->port_guid; - osm_log(p_vend->p_log, OSM_LOG_INFO, "osm_vendor_bind: " + OSM_LOG(p_vend->p_log, OSM_LOG_INFO, "Binding to port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); if ((umad_port_id = osm_vendor_open_port(p_vend, port_guid)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5424: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5424: " "Unable to open port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); goto Exit; @@ -822,16 +795,14 @@ osm_vendor_bind(IN osm_vendor_t * const p_vend, p_vend->umad_port.portnum, p_vend->issm_path, sizeof(p_vend->issm_path)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 542E: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 542E: " "Cannot resolve issm path for port %s:%u\n", p_vend->umad_port.ca_name, p_vend->umad_port.portnum); goto Exit; } if (!(p_bind = malloc(sizeof(*p_bind)))) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5425: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5425: " "Unable to allocate internal bind object\n"); goto Exit; } @@ -879,8 +850,7 @@ osm_vendor_bind(IN osm_vendor_t * const p_vend, p_user_bind->mad_class, p_user_bind->class_version, rmpp_version, method_mask)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5426: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5426: " "Unable to register class %u version %u\n", p_user_bind->mad_class, p_user_bind->class_version); free(p_bind); @@ -890,8 +860,7 @@ osm_vendor_bind(IN osm_vendor_t * const p_vend, if (p_bind->agent_id >= UMAD_CA_MAX_AGENTS || p_vend->agents[p_bind->agent_id]) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5427: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5427: " "bad agent id %u or duplicate agent for class %u vers %u\n", p_bind->agent_id, p_user_bind->mad_class, p_user_bind->class_version); @@ -909,8 +878,7 @@ osm_vendor_bind(IN osm_vendor_t * const p_vend, p_user_bind-> class_version, 0, method_mask)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5428: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5428: " "Unable to register class 1 version %u\n", p_user_bind->class_version); free(p_bind); @@ -920,8 +888,7 @@ osm_vendor_bind(IN osm_vendor_t * const p_vend, if (p_bind->agent_id1 >= UMAD_CA_MAX_AGENTS || p_vend->agents[p_bind->agent_id1]) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5429: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5429: " "bad agent id %u or duplicate agent for class 1 vers %u\n", p_bind->agent_id1, p_user_bind->class_version); free(p_bind); @@ -989,7 +956,7 @@ ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, OSM_LOG_ENTER(p_vend->p_log); - osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_get: " + OSM_LOG(p_vend->p_log, OSM_LOG_DEBUG, "Acquiring UMAD for p_madw = %p, size = %u\n", p_vw, mad_size); CL_ASSERT(p_vw); p_vw->size = mad_size; @@ -998,7 +965,7 @@ ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, /* track locally */ p_vw->h_bind = h_bind; - osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_get: " + OSM_LOG(p_vend->p_log, OSM_LOG_DEBUG, "Acquired UMAD %p, size = %u\n", p_vw->umad, p_vw->size); OSM_LOG_EXIT(p_vend->p_log); @@ -1018,8 +985,7 @@ osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) CL_ASSERT(p_vw); - osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_put: " - "Retiring UMAD %p\n", p_vw->umad); + OSM_LOG(p_vend->p_log, OSM_LOG_DEBUG, "Retiring UMAD %p\n", p_vw->umad); /* * We moved the removal of the transaction to immediately after @@ -1087,8 +1053,7 @@ osm_vendor_send(IN osm_bind_handle_t h_bind, #ifdef VENDOR_RMPP_SUPPORT } else is_rmpp = 1; - osm_log(p_vend->p_log, OSM_LOG_VERBOSE, - "osm_vendor_send: RMPP %d length %d\n", + OSM_LOG(p_vend->p_log, OSM_LOG_VERBOSE, "RMPP %d length %d\n", ib_rmpp_is_flag_set((ib_rmpp_mad_t *) p_sa, IB_RMPP_FLAG_ACTIVE), p_madw->mad_size); @@ -1119,8 +1084,7 @@ Resp: sent_mad_size, resp_expected ? p_vend->timeout : 0, p_vend->max_retries)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 5430: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5430: " "Send p_madw = %p of size %d failed %d (%m)\n", p_madw, sent_mad_size, ret); if (resp_expected) { @@ -1137,8 +1101,7 @@ Resp: if (!resp_expected) osm_mad_pool_put(p_bind->p_mad_pool, p_madw); - osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_send: " - "Completed sending %s p_madw = %p\n", + OSM_LOG(p_vend->p_log, OSM_LOG_DEBUG, "Completed sending %s p_madw = %p\n", resp_expected ? "request" : "response or unsolicited", p_madw); Exit: OSM_LOG_EXIT(p_vend->p_log); @@ -1169,8 +1132,7 @@ void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) if (TRUE == is_sm_val) { p_vend->issmfd = open(p_vend->issm_path, O_NONBLOCK); if (p_vend->issmfd < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 5431: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5431: " "setting IS_SM capmask: cannot open file " "\'%s\': %s\n", p_vend->issm_path, strerror(errno)); @@ -1178,8 +1140,7 @@ void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) } } else if (p_vend->issmfd != -1) { if (0 != close(p_vend->issmfd)) - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 5432: " + OSM_LOG(p_vend->p_log, OSM_LOG_ERROR, "ERR 5432: " "clearing IS_SM capmask: cannot close: %s\n", strerror(errno)); p_vend->issmfd = -1; diff --git a/opensm/libvendor/osm_vendor_ibumad_sa.c b/opensm/libvendor/osm_vendor_ibumad_sa.c index cef20aa..b9a1923 100644 --- a/opensm/libvendor/osm_vendor_ibumad_sa.c +++ b/opensm/libvendor/osm_vendor_ibumad_sa.c @@ -82,8 +82,7 @@ __osmv_sa_mad_rcv_cb(IN osm_madw_t * p_madw, OSM_LOG_ENTER(p_bind->p_log); if (!p_req_madw) { - osm_log(p_bind->p_log, OSM_LOG_DEBUG, - "__osmv_sa_mad_rcv_cb: " + OSM_LOG(p_bind->p_log, OSM_LOG_DEBUG, "Ignoring a non-response mad\n"); osm_mad_pool_put(p_bind->p_mad_pool, p_madw); goto Exit; @@ -106,8 +105,7 @@ __osmv_sa_mad_rcv_cb(IN osm_madw_t * p_madw, /* if we got a remote error track it in the status */ mad_status = (ib_net16_t) (p_sa_mad->status & IB_SMP_STATUS_MASK); if (mad_status != IB_SUCCESS) { - osm_log(p_bind->p_log, OSM_LOG_ERROR, - "__osmv_sa_mad_rcv_cb: ERR 5501: " + OSM_LOG(p_bind->p_log, OSM_LOG_ERROR, "ERR 5501: " "Remote error:0x%04X\n", mad_status); query_res.status = IB_REMOTE_ERROR; } else @@ -115,8 +113,7 @@ __osmv_sa_mad_rcv_cb(IN osm_madw_t * p_madw, /* what if we have got back an empty mad ? */ if (!p_madw->mad_size) { - osm_log(p_bind->p_log, OSM_LOG_ERROR, - "__osmv_sa_mad_rcv_cb: ERR 5502: " + OSM_LOG(p_bind->p_log, OSM_LOG_ERROR, "ERR 5502: " "Got an empty mad\n"); query_res.status = IB_ERROR; } @@ -140,8 +137,8 @@ __osmv_sa_mad_rcv_cb(IN osm_madw_t * p_madw, query_res.result_cnt = (uintn_t) ((p_madw->mad_size - IB_SA_MAD_HDR_SIZE) / ib_get_attr_size(p_sa_mad->attr_offset)); - osm_log(p_bind->p_log, OSM_LOG_DEBUG, - "__osmv_sa_mad_rcv_cb: Count = %u = %zu / %u (%zu)\n", + OSM_LOG(p_bind->p_log, OSM_LOG_DEBUG, + "Count = %u = %zu / %u (%zu)\n", query_res.result_cnt, p_madw->mad_size - IB_SA_MAD_HDR_SIZE, ib_get_attr_size(p_sa_mad->attr_offset), @@ -242,8 +239,7 @@ osmv_bind_sa(IN osm_vendor_t * const p_vend, OSM_LOG_ENTER(p_log); - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_bind_sa: " + OSM_LOG(p_log, OSM_LOG_DEBUG, "Binding to port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); bind_info.port_guid = port_guid; @@ -259,8 +255,7 @@ osmv_bind_sa(IN osm_vendor_t * const p_vend, p_sa_bind_info = (osmv_sa_bind_info_t *) malloc(sizeof(osmv_sa_bind_info_t)); if (!p_sa_bind_info) { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5505: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5505: " "Failed to allocate new bind structure\n"); p_sa_bind_info = OSM_BIND_INVALID_HANDLE; goto Exit; @@ -277,8 +272,7 @@ osmv_bind_sa(IN osm_vendor_t * const p_vend, if (p_sa_bind_info->h_bind == OSM_BIND_INVALID_HANDLE) { free(p_sa_bind_info); p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - osm_log(p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5506: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5506: " "Failed to bind to vendor GSI\n"); goto Exit; } @@ -290,8 +284,7 @@ osmv_bind_sa(IN osm_vendor_t * const p_vend, cl_event_construct(&p_sa_bind_info->sync_event); cl_status = cl_event_init(&p_sa_bind_info->sync_event, TRUE); if (cl_status != CL_SUCCESS) { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5508: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5508: " "cl_init_event failed: %s\n", ib_get_err_str(cl_status)); free(p_sa_bind_info); p_sa_bind_info = OSM_BIND_INVALID_HANDLE; @@ -372,8 +365,7 @@ __osmv_send_sa_req(IN osmv_sa_bind_info_t * p_bind, if (time(NULL) > p_bind->last_lids_update_sec + 30) { status = update_umad_port(p_bind->p_vendor); if (status != IB_SUCCESS) { - osm_log(p_log, OSM_LOG_ERROR, - "__osmv_send_sa_req: ERR 5509: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5509: " "Failed to obtain the SM lid\n"); goto Exit; } @@ -385,8 +377,7 @@ __osmv_send_sa_req(IN osmv_sa_bind_info_t * p_bind, p_bind->h_bind, MAD_BLOCK_SIZE, NULL); if (p_madw == NULL) { - osm_log(p_log, OSM_LOG_ERROR, - "__osmv_send_sa_req: ERR 5510: " + OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 5510: " "Unable to acquire MAD\n"); status = IB_INSUFFICIENT_RESOURCES; goto Exit; @@ -459,8 +450,7 @@ __osmv_send_sa_req(IN osmv_sa_bind_info_t * p_bind, /* if synchronous - wait on the event */ if (sync) { - osm_log(p_log, OSM_LOG_DEBUG, - "__osmv_send_sa_req: Waiting for async event\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "Waiting for async event\n"); cl_event_wait_on(&p_bind->sync_event, EVENT_NO_TIMEOUT, FALSE); cl_event_reset(&p_bind->sync_event); status = p_madw->status; @@ -506,8 +496,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, switch (p_query_req->query_type) { case OSMV_QUERY_USER_DEFINED: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "USER_DEFINED\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 USER_DEFINED\n"); p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; if (p_user_query->method) sa_mad_data.method = p_user_query->method; @@ -519,8 +508,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_ALL_SVC_RECS: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 SVC_REC_BY_NAME\n"); sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_service_record_t)); @@ -529,8 +517,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_SVC_REC_BY_NAME: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 SVC_REC_BY_NAME\n"); sa_mad_data.method = IB_MAD_METHOD_GET; sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; sa_mad_data.comp_mask = IB_SR_COMPMASK_SNAME; @@ -542,8 +529,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_SVC_REC_BY_ID: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_ID\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 SVC_REC_BY_ID\n"); sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; sa_mad_data.comp_mask = IB_SR_COMPMASK_SID; sa_mad_data.attr_offset = @@ -554,8 +540,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_CLASS_PORT_INFO: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "CLASS_PORT_INFO\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 CLASS_PORT_INFO\n"); sa_mad_data.method = IB_MAD_METHOD_GET; sa_mad_data.attr_id = IB_MAD_ATTR_CLASS_PORT_INFO; sa_mad_data.attr_offset = @@ -565,8 +550,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_NODE_REC_BY_NODE_GUID: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "NODE_REC_BY_NODE_GUID\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 NODE_REC_BY_NODE_GUID\n"); sa_mad_data.attr_id = IB_MAD_ATTR_NODE_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_node_record_t)); @@ -577,8 +561,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_PORT_REC_BY_LID: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "PORT_REC_BY_LID\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 PORT_REC_BY_LID\n"); sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_portinfo_record_t)); @@ -590,9 +573,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, case OSMV_QUERY_PORT_REC_BY_LID_AND_NUM: sa_mad_data.method = IB_MAD_METHOD_GET; p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", - "PORT_REC_BY_LID_AND_NUM\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 PORT_REC_BY_LID_AND_NUM\n"); sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_portinfo_record_t)); @@ -604,9 +585,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, case OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK: sa_mad_data.method = IB_MAD_METHOD_GET; p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", - "OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); sa_mad_data.attr_id = IB_MAD_ATTR_VLARB_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_vl_arb_table_record_t)); @@ -619,9 +598,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, case OSMV_QUERY_SLVL_BY_LID_AND_PORTS: sa_mad_data.method = IB_MAD_METHOD_GET; p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", - "OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); sa_mad_data.attr_id = IB_MAD_ATTR_SLVL_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_slvl_table_record_t)); @@ -632,8 +609,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_PATH_REC_BY_PORT_GUIDS: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "PATH_REC_BY_PORT_GUIDS\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 PATH_REC_BY_PORT_GUIDS\n"); memset(&path_rec, 0, sizeof(ib_path_rec_t)); sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; sa_mad_data.attr_offset = @@ -652,8 +628,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_PATH_REC_BY_GIDS: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "PATH_REC_BY_GIDS\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 PATH_REC_BY_GIDS\n"); memset(&path_rec, 0, sizeof(ib_path_rec_t)); sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; sa_mad_data.attr_offset = @@ -670,8 +645,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, break; case OSMV_QUERY_PATH_REC_BY_LIDS: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "PATH_REC_BY_LIDS\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 PATH_REC_BY_LIDS\n"); memset(&path_rec, 0, sizeof(ib_path_rec_t)); sa_mad_data.method = IB_MAD_METHOD_GET; sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; @@ -690,9 +664,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, case OSMV_QUERY_UD_MULTICAST_SET: sa_mad_data.method = IB_MAD_METHOD_SET; p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", - "OSMV_QUERY_UD_MULTICAST_SET\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 OSMV_QUERY_UD_MULTICAST_SET\n"); sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_member_rec_t)); @@ -703,9 +675,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, case OSMV_QUERY_UD_MULTICAST_DELETE: sa_mad_data.method = IB_MAD_METHOD_DELETE; p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", - "OSMV_QUERY_UD_MULTICAST_DELETE\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 OSMV_QUERY_UD_MULTICAST_DELETE\n"); sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; sa_mad_data.attr_offset = ib_get_attr_offset(sizeof(ib_member_rec_t)); @@ -715,14 +685,12 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, #ifdef DUAL_SIDED_RMPP case OSMV_QUERY_MULTIPATH_REC: - osm_log(p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "MULTIPATH_REC\n"); + OSM_LOG(p_log, OSM_LOG_DEBUG, "DBG:001 MULTIPATH_REC\n"); /* Validate sgid/dgid counts against SA client limit */ p_mpr_req = (osmv_multipath_req_t *) p_query_req->p_query_input; if (p_mpr_req->sgid_count + p_mpr_req->dgid_count > IB_MULTIPATH_MAX_GIDS) { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_query_sa DBG:001 MULTIPATH_REC " + OSM_LOG(p_log, OSM_LOG_ERROR, "DBG:001 MULTIPATH_REC " "SGID count %d DGID count %d max count %d\n", p_mpr_req->sgid_count, p_mpr_req->dgid_count, IB_MULTIPATH_MAX_GIDS); @@ -756,8 +724,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind, #endif default: - osm_log(p_log, OSM_LOG_ERROR, - "osmv_query_sa DBG:001 %s", "UNKNOWN\n"); + OSM_LOG(p_log, OSM_LOG_ERROR, "DBG:001 UNKNOWN\n"); CL_ASSERT(0); return IB_ERROR; } -- 1.5.5.1.178.g1f811 From ronli at voltaire.com Sun Jul 27 03:25:52 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 27 Jul 2008 10:25:52 +0000 (UTC) Subject: [ofa-general] [PATCH 1/1 v2] librdmacm: add support for create qp expanded Message-ID: diff --git a/include/rdma/rdma_cma.h b/include/rdma/rdma_cma.h index a516ab8..34c6b9f 100644 --- a/include/rdma/rdma_cma.h +++ b/include/rdma/rdma_cma.h @@ -296,6 +296,10 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms); int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); +int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags); + /** * rdma_destroy_qp - Deallocate a QP. * @id: RDMA identifier. diff --git a/src/cma.c b/src/cma.c index ecb41bc..4e2da76 100644 --- a/src/cma.c +++ b/src/cma.c @@ -783,33 +783,49 @@ static int ucma_init_ud_qp(struct cma_id_private *id_priv, struct ibv_qp *qp) return ibv_modify_qp(qp, &qp_attr, IBV_QP_STATE | IBV_QP_SQ_PSN); } -int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, - struct ibv_qp_init_attr *qp_init_attr) +static int rdma_create_qp_common(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags, int expanded) { - struct cma_id_private *id_priv; - struct ibv_qp *qp; - int ret; - - id_priv = container_of(id, struct cma_id_private, id); - if (id->verbs != pd->context) - return -EINVAL; - - qp = ibv_create_qp(pd, qp_init_attr); - if (!qp) - return -ENOMEM; + struct cma_id_private *id_priv; + struct ibv_qp *qp; + int ret; + + id_priv = container_of(id, struct cma_id_private, id); + if (id->verbs != pd->context) + return -EINVAL; + + qp = expanded ? + ibv_create_qp_expanded(pd, qp_init_attr, create_flags) : + ibv_create_qp(pd, qp_init_attr); + if (!qp) + return -ENOMEM; + + if (ucma_is_ud_ps(id->ps)) + ret = ucma_init_ud_qp(id_priv, qp); + else + ret = ucma_init_conn_qp(id_priv, qp); + if (ret) + goto err; + + id->qp = qp; + return 0; +err: + ibv_destroy_qp(qp); + return ret; +} - if (ucma_is_ud_ps(id->ps)) - ret = ucma_init_ud_qp(id_priv, qp); - else - ret = ucma_init_conn_qp(id_priv, qp); - if (ret) - goto err; +int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags) +{ + return rdma_create_qp_common(id, pd, qp_init_attr, create_flags, 1); +} - id->qp = qp; - return 0; -err: - ibv_destroy_qp(qp); - return ret; +int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr) +{ + return rdma_create_qp_common(id, pd, qp_init_attr, 0, 0); } void rdma_destroy_qp(struct rdma_cm_id *id) diff --git a/src/librdmacm.map b/src/librdmacm.map index cb94efe..b237eda 100644 --- a/src/librdmacm.map +++ b/src/librdmacm.map @@ -28,5 +28,6 @@ RDMACM_1.0 { rdma_get_local_addr; rdma_get_peer_addr; rdma_migrate_id; + rdma_create_qp_expanded; local: *; }; From ronli at voltaire.com Sun Jul 27 03:31:29 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 27 Jul 2008 10:31:29 +0000 (UTC) Subject: [ofa-general] [PATCH 1/1 v2] librdmacm: add support for create qp expanded (with changelog this time) Message-ID: Adds a new function: int rdma_create_qp_expanded which uses the ibv_create_qp_expanded function in libibverbs and uses it similarly to ibv_create_qp, with the difference of creation flags. Changes in v2: Added compatibility to old libibverbs. Signed-off-by: Ron Livne diff --git a/include/rdma/rdma_cma.h b/include/rdma/rdma_cma.h index a516ab8..34c6b9f 100644 --- a/include/rdma/rdma_cma.h +++ b/include/rdma/rdma_cma.h @@ -296,6 +296,10 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms); int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); +int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags); + /** * rdma_destroy_qp - Deallocate a QP. * @id: RDMA identifier. diff --git a/src/cma.c b/src/cma.c index ecb41bc..4e2da76 100644 --- a/src/cma.c +++ b/src/cma.c @@ -783,33 +783,49 @@ static int ucma_init_ud_qp(struct cma_id_private *id_priv, struct ibv_qp *qp) return ibv_modify_qp(qp, &qp_attr, IBV_QP_STATE | IBV_QP_SQ_PSN); } -int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, - struct ibv_qp_init_attr *qp_init_attr) +static int rdma_create_qp_common(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags, int expanded) { - struct cma_id_private *id_priv; - struct ibv_qp *qp; - int ret; - - id_priv = container_of(id, struct cma_id_private, id); - if (id->verbs != pd->context) - return -EINVAL; - - qp = ibv_create_qp(pd, qp_init_attr); - if (!qp) - return -ENOMEM; + struct cma_id_private *id_priv; + struct ibv_qp *qp; + int ret; + + id_priv = container_of(id, struct cma_id_private, id); + if (id->verbs != pd->context) + return -EINVAL; + + qp = expanded ? + ibv_create_qp_expanded(pd, qp_init_attr, create_flags) : + ibv_create_qp(pd, qp_init_attr); + if (!qp) + return -ENOMEM; + + if (ucma_is_ud_ps(id->ps)) + ret = ucma_init_ud_qp(id_priv, qp); + else + ret = ucma_init_conn_qp(id_priv, qp); + if (ret) + goto err; + + id->qp = qp; + return 0; +err: + ibv_destroy_qp(qp); + return ret; +} - if (ucma_is_ud_ps(id->ps)) - ret = ucma_init_ud_qp(id_priv, qp); - else - ret = ucma_init_conn_qp(id_priv, qp); - if (ret) - goto err; +int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags) +{ + return rdma_create_qp_common(id, pd, qp_init_attr, create_flags, 1); +} - id->qp = qp; - return 0; -err: - ibv_destroy_qp(qp); - return ret; +int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr) +{ + return rdma_create_qp_common(id, pd, qp_init_attr, 0, 0); } void rdma_destroy_qp(struct rdma_cm_id *id) diff --git a/src/librdmacm.map b/src/librdmacm.map index cb94efe..b237eda 100644 --- a/src/librdmacm.map +++ b/src/librdmacm.map @@ -28,5 +28,6 @@ RDMACM_1.0 { rdma_get_local_addr; rdma_get_peer_addr; rdma_migrate_id; + rdma_create_qp_expanded; local: *; }; From jackm at dev.mellanox.co.il Sun Jul 27 00:50:55 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 27 Jul 2008 10:50:55 +0300 Subject: [ofa-general] [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) In-Reply-To: References: <200807151745.53152.jackm@dev.mellanox.co.il> Message-ID: <200807271050.55869.jackm@dev.mellanox.co.il> On Monday 21 July 2008 19:46, Hal Rosenstock wrote: > Out of curiousity, why is a module parameter needed to enable/disable > QoS ? Is there a downside to enabling it in terms of the HCA ? > > -- Hal > We noticed (after ofed 1.3 was released) that setting the QoS bit in the HCA resulted in using more HW resources. Since Qos is usually not used we decided to disable the feature by default, and allow enabling it when it is needed. In ofed 1.3.1 the default value was therefore changed to be OFF. - Jack From sashak at voltaire.com Sun Jul 27 01:47:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 11:47:08 +0300 Subject: [ofa-general] [PATCHi v2] opensm/osm_ucast_lash: find_port_from_lid() function is not used In-Reply-To: <20080726174259.GH5067@sashak.voltaire.com> References: <20080726174025.GF5067@sashak.voltaire.com> <20080726174123.GG5067@sashak.voltaire.com> <20080726174259.GH5067@sashak.voltaire.com> Message-ID: <20080727084708.GP5067@sashak.voltaire.com> From: Keshetti Mahesh This function is called only for the case when source and destination switches are the same, so it is routing to itself or to CA connected to this switch. The patch removes find_port_from_lid() function and its usage. Signed-off-by: Sasha Khapyorsky --- Change from v1 is that not only switch itself but also its CAs are handled - as was originally. opensm/opensm/osm_ucast_lash.c | 73 ++++----------------------------------- 1 files changed, 8 insertions(+), 65 deletions(-) diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c index bb929b6..c7db542 100644 --- a/opensm/opensm/osm_ucast_lash.c +++ b/opensm/opensm/osm_ucast_lash.c @@ -145,67 +145,6 @@ static osm_switch_t *get_osm_switch_from_port(osm_port_t * port) return NULL; } -static osm_switch_t *get_osm_switch_from_lid(osm_opensm_t * osm, uint16_t lid) -{ - osm_port_t *port = cl_ptr_vector_get(&osm->subn.port_lid_tbl, lid); - if (!port) - return NULL; - return get_osm_switch_from_port(port); -} - -// This is a time consuming way to find a port from a lid -// we will come up with a better way later -static uint8_t find_port_from_lid(IN const ib_net16_t lid_no, - IN const osm_switch_t * p_sw) -{ - uint8_t port_count = 0; - uint8_t i = 0; - osm_physp_t *p_current_physp, *p_remote_physp = NULL; - ib_port_info_t *port_info; - ib_net16_t port_lid; - uint8_t egress_port = 255; - - if (p_sw->p_node) - port_count = osm_node_get_num_physp(p_sw->p_node); - - // process management port first - p_current_physp = osm_node_get_physp_ptr(p_sw->p_node, 0); - - port_info = &p_current_physp->port_info; - port_lid = port_info->base_lid; - if (port_lid == lid_no) { - egress_port = 0; - goto Exit; - } - // process each port on this switch - for (i = 1; i < port_count; i++) { - - p_current_physp = osm_node_get_physp_ptr(p_sw->p_node, i); - if (!p_current_physp) - continue; - - p_remote_physp = p_current_physp->p_remote_physp; - - if (p_remote_physp) { - osm_node_t *p_opposite_node = - osm_physp_get_node_ptr(p_remote_physp); - - if (osm_node_get_type(p_opposite_node) == - IB_NODE_TYPE_CA) { - ib_port_info_t *pi = &p_remote_physp->port_info; - ib_net16_t remote_port_lid = pi->base_lid; - if (remote_port_lid == lid_no) { - egress_port = i; - goto Exit; - } - } - } - } // for - -Exit: - return egress_port; -} - #if 0 static int randint(int high) { @@ -1111,7 +1050,8 @@ static void populate_fwd_tbls(lash_t * p_lash) osm_subn_t *p_subn = &p_lash->p_osm->subn; osm_opensm_t *p_osm = p_lash->p_osm; osm_switch_t *p_sw, *p_next_sw, *p_dst_sw; - uint16_t max_lid_ho, lid = 0; + osm_port_t *port; + uint16_t max_lid_ho, lid; OSM_LOG_ENTER(p_log); @@ -1132,11 +1072,14 @@ static void populate_fwd_tbls(lash_t * p_lash) IB_LID_UCAST_END_HO + 1); for (lid = 1; lid <= max_lid_ho; lid++) { - p_dst_sw = get_osm_switch_from_lid(p_lash->p_osm, lid); + port = cl_ptr_vector_get(&p_subn->port_lid_tbl, lid); + if (!port) + continue; + p_dst_sw = get_osm_switch_from_port(port); if (p_dst_sw == p_sw) { - uint8_t egress_port = - find_port_from_lid(cl_hton16(lid), p_sw); + uint8_t egress_port = port->p_node->sw ? 0 : + port->p_physp->p_remote_physp->port_num; p_osm->sm.ucast_mgr.lft_buf[lid] = egress_port; OSM_LOG(p_log, OSM_LOG_VERBOSE, "LASH fwd MY SRC SRC GUID 0x%016" PRIx64 -- 1.5.5.1.178.g1f811 From vlad at lists.openfabrics.org Sun Jul 27 02:52:47 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sun, 27 Jul 2008 02:52:47 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080727-0200 daily build status Message-ID: <20080727095248.18207E60316@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: Build failed on ia64 with linux-2.6.23 Log: from /home/vlad/tmp/ofa_1_4_kernel-20080727-0200_linux-2.6.23_ia64_check/kernel_addons/backport/2.6.23/include/linux/inetdevice.h:4, from /home/vlad/tmp/ofa_1_4_kernel-20080727-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.c:32: include/linux/slub_def.h:50: error: field 'kobj' has incomplete type make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080727-0200_linux-2.6.23_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080727-0200_linux-2.6.23_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20080727-0200_linux-2.6.23_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20080727-0200_linux-2.6.23_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.23' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From tziporet at dev.mellanox.co.il Sun Jul 27 04:24:37 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 27 Jul 2008 14:24:37 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.4 alpha release is ready In-Reply-To: <488B4990.2010208@opengridcomputing.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> <488B4990.2010208@opengridcomputing.com> Message-ID: <488C5AF5.1070002@mellanox.co.il> Steve Wise wrote: > How can we claim an alpha release when the tree is still based on > linux-2.6.26 and doesn't have a large chunk of the new features > integrated? > This was the process we decided to follow at Sonoma. I know some features are not there yet, but many already in and starting moving the release process forward is important even if not all features are in. Tziporet From ronli at voltaire.com Sun Jul 27 07:33:35 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 27 Jul 2008 14:33:35 +0000 (UTC) Subject: [ofa-general] [PATCH 2/3 v3] libmlx4: add support for create_qp_expanded Message-ID: Adds support for the new verb: create_qp_expanded in libmlx4. Changes in v2: Minimized code duplications by calling mlx4_create_qp_expanded from mlx4_create_qp. Changes in v3: Minimized code duplications by calling mlx4_create_qp_common. Added compatibility for older kernels. Signed-off-by: Ron Livne diff --git a/src/mlx4-abi.h b/src/mlx4-abi.h index 1b1253c..729f4ee 100644 --- a/src/mlx4-abi.h +++ b/src/mlx4-abi.h @@ -98,6 +98,16 @@ struct mlx4_create_qp { __u8 reserved[5]; }; +struct mlx4_create_qp_expanded { + struct ibv_create_qp_expanded ibv_cmd; + __u64 buf_addr; + __u64 db_addr; + __u8 log_sq_bb_count; + __u8 log_sq_stride; + __u8 sq_no_prefetch; + __u8 reserved[5]; +}; + #ifdef HAVE_IBV_XRC_OPS struct mlx4_open_xrc_domain_resp { struct ibv_open_xrc_domain_resp ibv_resp; diff --git a/src/mlx4.c b/src/mlx4.c index 1f23144..bfcb7ec 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -79,6 +79,7 @@ static struct ibv_more_ops mlx4_more_ops = { .query_xrc_rcv_qp = mlx4_query_xrc_rcv_qp, .reg_xrc_rcv_qp = mlx4_reg_xrc_rcv_qp, .unreg_xrc_rcv_qp = mlx4_unreg_xrc_rcv_qp, + .create_qp_expanded = mlx4_create_qp_expanded, #endif }; #endif diff --git a/src/mlx4.h b/src/mlx4.h index a2e3aec..7ed3ac8 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -427,6 +427,9 @@ int mlx4_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags); #endif diff --git a/src/verbs.c b/src/verbs.c index b1bad3c..1c4b662 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -402,23 +402,16 @@ int mlx4_destroy_srq(struct ibv_srq *ibsrq) return 0; } -struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) +static struct mlx4_qp *mlx4_create_qp_common(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + struct mlx4_qp *qp) { - struct mlx4_create_qp cmd; - struct ibv_create_qp_resp resp; - struct mlx4_qp *qp; - int ret; - /* Sanity check QP size before proceeding */ if (attr->cap.max_send_wr > 65536 || - attr->cap.max_recv_wr > 65536 || - attr->cap.max_send_sge > 64 || - attr->cap.max_recv_sge > 64 || - attr->cap.max_inline_data > 1024) - return NULL; - - qp = malloc(sizeof *qp); - if (!qp) + attr->cap.max_recv_wr > 65536 || + attr->cap.max_send_sge > 64 || + attr->cap.max_recv_sge > 64 || + attr->cap.max_inline_data > 1024) return NULL; mlx4_calc_sq_wqe_size(&attr->cap, attr->qp_type, qp); @@ -456,6 +449,32 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) *qp->db = 0; } + return qp; +err_free: + free(qp->sq.wrid); + if (qp->rq.wqe_cnt) + free(qp->rq.wrid); + mlx4_free_buf(&qp->buf); + +err: + free(qp); + + return NULL; +} + +struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) +{ + struct mlx4_create_qp cmd; + struct ibv_create_qp_resp resp; + struct mlx4_qp *qp; + int ret; + + qp = malloc(sizeof *qp); + if (!qp) + return NULL; + qp = mlx4_create_qp_common(pd, attr, qp); + if (!qp) + return NULL; cmd.buf_addr = (uintptr_t) qp->buf.buf; if (attr->srq || attr->qp_type == IBV_QPT_XRC) @@ -467,7 +486,7 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) qp->sq.wqe_cnt > 1 << cmd.log_sq_bb_count; ++cmd.log_sq_bb_count) ; /* nothing */ - cmd.sq_no_prefetch = 0; /* OK for ABI 2: just a reserved field */ + cmd.sq_no_prefetch = 0; /* OK for ABI 2: just a reserved field */ memset(cmd.reserved, 0, sizeof cmd.reserved); ret = ibv_cmd_create_qp(pd, &qp->ibv_qp, attr, &cmd.ibv_cmd, sizeof cmd, @@ -497,14 +516,11 @@ err_destroy: err_rq_db: if (!attr->srq && attr->qp_type != IBV_QPT_XRC) mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); - -err_free: free(qp->sq.wrid); if (qp->rq.wqe_cnt) free(qp->rq.wrid); mlx4_free_buf(&qp->buf); -err: free(qp); return NULL; @@ -825,4 +841,72 @@ int mlx4_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, return ibv_cmd_unreg_xrc_rcv_qp(xrc_domain, xrc_qp_num); } +struct ibv_qp *mlx4_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags) +{ + struct mlx4_create_qp_expanded cmd; + struct ibv_create_qp_resp resp; + struct mlx4_qp *qp; + int ret; + + qp = malloc(sizeof *qp); + if (!qp) + return NULL; + qp = mlx4_create_qp_common(pd, attr, qp); + if (!qp) + return NULL; + + cmd.buf_addr = (uintptr_t) qp->buf.buf; + if (attr->srq || attr->qp_type == IBV_QPT_XRC) + cmd.db_addr = 0; + else + cmd.db_addr = (uintptr_t) qp->db; + cmd.log_sq_stride = qp->sq.wqe_shift; + for (cmd.log_sq_bb_count = 0; + qp->sq.wqe_cnt > 1 << cmd.log_sq_bb_count; + ++cmd.log_sq_bb_count) + ; /* nothing */ + cmd.sq_no_prefetch = 0; + + ret = ibv_cmd_create_qp_expanded(pd, &qp->ibv_qp, attr, create_flags, + &cmd.ibv_cmd, + sizeof cmd, &resp, sizeof resp); + if (ret) + goto err_rq_db; + + ret = mlx4_store_qp(to_mctx(pd->context), qp->ibv_qp.qp_num, qp); + if (ret) + goto err_destroy; + + qp->rq.wqe_cnt = qp->rq.max_post = attr->cap.max_recv_wr; + qp->rq.max_gs = attr->cap.max_recv_sge; + mlx4_set_sq_sizes(qp, &attr->cap, attr->qp_type); + + qp->doorbell_qpn = htonl(qp->ibv_qp.qp_num << 8); + if (attr->sq_sig_all) + qp->sq_signal_bits = htonl(MLX4_WQE_CTRL_CQ_UPDATE); + else + qp->sq_signal_bits = 0; + + return &qp->ibv_qp; + +err_destroy: + ibv_cmd_destroy_qp(&qp->ibv_qp); + +err_rq_db: + if (!attr->srq && attr->qp_type != IBV_QPT_XRC) + mlx4_free_db(to_mctx(pd->context), MLX4_DB_TYPE_RQ, qp->db); + + free(qp->sq.wrid); + if (qp->rq.wqe_cnt) + free(qp->rq.wrid); + mlx4_free_buf(&qp->buf); + + free(qp); + + return NULL; +} + + #endif From ronli at voltaire.com Sun Jul 27 07:34:33 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 27 Jul 2008 14:34:33 +0000 (UTC) Subject: [ofa-general] [PATCH 1/3 v3] libibverbs : add new verbs: create_qp_expanded Message-ID: 1. This patch adds a new capability flags to the enum ibv_device_cap_flags IBV_DEVICE_BLOCK_MULTICAST_LOOPBACK. which implies that the device is capable of blocking multicast loopback packets. 2. This patch also adds a new verb to the libibverbs: struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr, uint32_t create_flags); which works similarly to ibv_create_qp, except for that it now takes another argument: uint32_t create_flags these creation flags should be aligned with those in the ib_verbs.h in the kernel. 3. New creation flags added: IBV_QP_CREATE_IPOIB_UD_LSO, IBV_MULTICAST_LOOPBACK_BLOCK Changes in v2: LSO creation flag deleted from user space. Added struct ibv_create_qp_expanded. Changes in v3: Added compatibility for old kernel. Added __u32 create_flags to ibv_create_qp_expanded. Signed-off-by: Ron Livne diff --git a/include/infiniband/driver.h b/include/infiniband/driver.h index f8138ef..ea824c8 100644 --- a/include/infiniband/driver.h +++ b/include/infiniband/driver.h @@ -155,6 +155,11 @@ int ibv_cmd_reg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); int ibv_cmd_unreg_xrc_rcv_qp(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + uint32_t create_flags, + struct ibv_create_qp_expanded *cmd, size_t cmd_size, + struct ibv_create_qp_resp *resp, size_t resp_size); /* * sysfs helper functions diff --git a/include/infiniband/kern-abi.h b/include/infiniband/kern-abi.h index 8b5cd9a..1d4aa95 100644 --- a/include/infiniband/kern-abi.h +++ b/include/infiniband/kern-abi.h @@ -94,6 +94,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, IB_USER_VERBS_CMD_REG_XRC_RCV_QP, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED, }; /* @@ -459,6 +460,30 @@ struct ibv_create_qp { __u64 driver_data[0]; }; +struct ibv_create_qp_expanded { + __u32 command; + __u16 in_words; + __u16 out_words; + __u64 response; + __u64 user_handle; + __u32 pd_handle; + __u32 send_cq_handle; + __u32 recv_cq_handle; + __u32 srq_handle; + __u32 max_send_wr; + __u32 max_recv_wr; + __u32 max_send_sge; + __u32 max_recv_sge; + __u32 max_inline_data; + __u8 sq_sig_all; + __u8 qp_type; + __u8 is_srq; + __u8 reserved; + __u32 reserved1; + __u32 create_flags; + __u64 driver_data[0]; +}; + struct ibv_create_qp_resp { __u32 qp_handle; __u32 qpn; @@ -945,6 +970,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP_V2 = -1, IB_USER_VERBS_CMD_REG_XRC_RCV_QP_V2 = -1, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP_V2 = -1, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED_V2 = -1, }; struct ibv_destroy_cq_v1 { diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index 2865d5c..b88aa7f 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -93,7 +93,8 @@ enum ibv_device_cap_flags { IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, IBV_DEVICE_SRQ_RESIZE = 1 << 13, IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, - IBV_DEVICE_XRC = 1 << 20 + IBV_DEVICE_XRC = 1 << 20, + IBV_DEVICE_BLOCK_MULTICAST_LOOPBACK = 1 << 22 }; enum ibv_atomic_cap { @@ -631,6 +632,13 @@ enum { IBV_SYSFS_PATH_MAX = 256 }; +/* + This enum must be alligned with ib_qp_create_flags + in include/rdma/ib_verbs.h +*/ +enum ibv_qp_create_flags { + IBV_QP_CREATE_MULTICAST_LOOPBACK_BLOCK = 1 << 1 +}; struct ibv_device { struct ibv_device_ops ops; enum ibv_node_type node_type; @@ -668,6 +676,9 @@ struct ibv_more_ops { uint32_t xrc_qp_num); int (*unreg_xrc_rcv_qp)(struct ibv_xrc_domain *xrc_domain, uint32_t xrc_qp_num); + struct ibv_qp * (*create_qp_expanded)(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr, + uint32_t create_flags); }; @@ -1029,6 +1040,13 @@ struct ibv_qp *ibv_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); /** + * ibv_create_qp_expanded - Create a queue pair with creation flags. + */ +struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags); + +/** * ibv_modify_qp - Modify a queue pair. */ int ibv_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, diff --git a/src/cmd.c b/src/cmd.c index cedf55e..45f7dd1 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -620,6 +620,45 @@ int ibv_cmd_destroy_srq(struct ibv_srq *srq) return 0; } +static int ibv_cmd_create_qp_common(struct ibv_pd *pd, + struct ibv_qp *qp, + struct ibv_qp_init_attr *attr, + struct ibv_create_qp_resp *resp, + size_t resp_size) +{ + VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); + + qp->handle = resp->qp_handle; + qp->qp_num = resp->qpn; + qp->context = pd->context; + + if (abi_ver > 3) { + attr->cap.max_recv_sge = resp->max_recv_sge; + attr->cap.max_send_sge = resp->max_send_sge; + attr->cap.max_recv_wr = resp->max_recv_wr; + attr->cap.max_send_wr = resp->max_send_wr; + attr->cap.max_inline_data = resp->max_inline_data; + } + + if (abi_ver == 4) { + struct ibv_create_qp_resp_v4 *resp_v4 = + (struct ibv_create_qp_resp_v4 *) resp; + + memmove((void *) resp + sizeof *resp, + (void *) resp_v4 + sizeof *resp_v4, + resp_size - sizeof *resp); + } else if (abi_ver <= 3) { + struct ibv_create_qp_resp_v3 *resp_v3 = + (struct ibv_create_qp_resp_v3 *) resp; + + memmove((void *) resp + sizeof *resp, + (void *) resp_v3 + sizeof *resp_v3, + resp_size - sizeof *resp); + } + + return 0; +} + int ibv_cmd_create_qp(struct ibv_pd *pd, struct ibv_qp *qp, struct ibv_qp_init_attr *attr, struct ibv_create_qp *cmd, size_t cmd_size, @@ -627,6 +666,7 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, { IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP, resp, resp_size); + cmd->reserved = 0; cmd->user_handle = (uintptr_t) qp; cmd->pd_handle = pd->handle; cmd->send_cq_handle = attr->send_cq->handle; @@ -642,42 +682,42 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? (attr->xrc_domain ? attr->xrc_domain->handle : 0) : (attr->srq ? attr->srq->handle : 0); - cmd->reserved = 0; if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; - VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); - - qp->handle = resp->qp_handle; - qp->qp_num = resp->qpn; - qp->context = pd->context; - - if (abi_ver > 3) { - attr->cap.max_recv_sge = resp->max_recv_sge; - attr->cap.max_send_sge = resp->max_send_sge; - attr->cap.max_recv_wr = resp->max_recv_wr; - attr->cap.max_send_wr = resp->max_send_wr; - attr->cap.max_inline_data = resp->max_inline_data; - } + return ibv_cmd_create_qp_common(pd, qp, attr, resp, resp_size); +} - if (abi_ver == 4) { - struct ibv_create_qp_resp_v4 *resp_v4 = - (struct ibv_create_qp_resp_v4 *) resp; +int ibv_cmd_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + uint32_t create_flags, + struct ibv_create_qp_expanded *cmd, size_t cmd_size, + struct ibv_create_qp_resp *resp, size_t resp_size) +{ + IBV_INIT_CMD_RESP(cmd, cmd_size, CREATE_QP_EXPANDED, resp, resp_size); - memmove((void *) resp + sizeof *resp, - (void *) resp_v4 + sizeof *resp_v4, - resp_size - sizeof *resp); - } else if (abi_ver <= 3) { - struct ibv_create_qp_resp_v3 *resp_v3 = - (struct ibv_create_qp_resp_v3 *) resp; + cmd->create_flags = create_flags; + cmd->user_handle = (uintptr_t) qp; + cmd->pd_handle = pd->handle; + cmd->send_cq_handle = attr->send_cq->handle; + cmd->recv_cq_handle = attr->recv_cq->handle; + cmd->max_send_wr = attr->cap.max_send_wr; + cmd->max_recv_wr = attr->cap.max_recv_wr; + cmd->max_send_sge = attr->cap.max_send_sge; + cmd->max_recv_sge = attr->cap.max_recv_sge; + cmd->max_inline_data = attr->cap.max_inline_data; + cmd->sq_sig_all = attr->sq_sig_all; + cmd->qp_type = attr->qp_type; + cmd->is_srq = !!attr->srq; + cmd->srq_handle = attr->qp_type == IBV_QPT_XRC ? + (attr->xrc_domain ? attr->xrc_domain->handle : 0) : + (attr->srq ? attr->srq->handle : 0); - memmove((void *) resp + sizeof *resp, - (void *) resp_v3 + sizeof *resp_v3, - resp_size - sizeof *resp); - } + if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) + return errno; - return 0; + return ibv_cmd_create_qp_common(pd, qp, attr, resp, resp_size); } int ibv_cmd_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, diff --git a/src/libibverbs.map b/src/libibverbs.map index fce6965..c032779 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -112,4 +112,6 @@ IBVERBS_1.1 { ibv_port_state_str; ibv_event_type_str; ibv_wc_status_str; + ibv_create_qp_expanded; + ibv_cmd_create_qp_expanded; } IBVERBS_1.0; diff --git a/src/verbs.c b/src/verbs.c index 4bfeec2..f185306 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -424,20 +424,19 @@ int __ibv_destroy_srq(struct ibv_srq *srq) } default_symver(__ibv_destroy_srq, ibv_destroy_srq); -struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, - struct ibv_qp_init_attr *qp_init_attr) +static struct ibv_qp *create_qp_common(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + struct ibv_qp *qp) { - struct ibv_qp *qp = pd->context->ops.create_qp(pd, qp_init_attr); - if (qp) { - qp->context = pd->context; - qp->qp_context = qp_init_attr->qp_context; - qp->pd = pd; - qp->send_cq = qp_init_attr->send_cq; - qp->recv_cq = qp_init_attr->recv_cq; - qp->srq = qp_init_attr->srq; + qp->context = pd->context; + qp->qp_context = qp_init_attr->qp_context; + qp->pd = pd; + qp->send_cq = qp_init_attr->send_cq; + qp->recv_cq = qp_init_attr->recv_cq; + qp->srq = qp_init_attr->srq; qp->qp_type = qp_init_attr->qp_type; - qp->state = IBV_QPS_RESET; + qp->state = IBV_QPS_RESET; qp->events_completed = 0; qp->xrc_domain = qp_init_attr->qp_type == IBV_QPT_XRC ? qp_init_attr->xrc_domain : NULL; @@ -447,8 +446,26 @@ struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, return qp; } + +struct ibv_qp *__ibv_create_qp(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr) +{ + + struct ibv_qp *qp = pd->context->ops.create_qp(pd, qp_init_attr); + + return create_qp_common(pd, qp_init_attr, qp); +} default_symver(__ibv_create_qp, ibv_create_qp); +struct ibv_qp *ibv_create_qp_expanded(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr, + uint32_t create_flags) +{ + struct ibv_qp *qp = pd->context->more_ops->create_qp_expanded(pd, qp_init_attr, create_flags); + + return create_qp_common(pd, qp_init_attr, qp); +} + int __ibv_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, enum ibv_qp_attr_mask attr_mask, struct ibv_qp_init_attr *init_attr) From tziporet at dev.mellanox.co.il Sun Jul 27 04:29:50 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 27 Jul 2008 14:29:50 +0300 Subject: [ofa-general] Problem with ipath_dma.c and dma-mapping.h In-Reply-To: References: <200807250539.m6P5dWc04020@gandalf.iuzzolino.com> Message-ID: <488C5C2E.4060802@mellanox.co.il> Roland Dreier wrote: > > BUT I shouldn't have to do fixes like that. And the question is, if > > this is a 64 bit machine, and the dma-mapping.h defines valid_dma_direction > > ONLY IF you are NOT compiling for a 64 bit machine, then the Openfabrics > > files: ipath_dma.c and/or dma-mapping.h have some kind of mistake. > > It's a broken backport. valid_dma_direction() was moved into generic > code between 2.6.18 and 2.6.19, so someone needs to add a backport patch > to handle kernels 2.6.18 and older. > > Qlogic people Please fix the broken backport Thanks Tziporet From hal.rosenstock at gmail.com Sun Jul 27 04:40:00 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 27 Jul 2008 07:40:00 -0400 Subject: [ofa-general] [PATCH] mlx4_core: add module parameter to enable/disable QoS support. (for 2.6.27) In-Reply-To: <200807271050.55869.jackm@dev.mellanox.co.il> References: <200807151745.53152.jackm@dev.mellanox.co.il> <200807271050.55869.jackm@dev.mellanox.co.il> Message-ID: On Sun, Jul 27, 2008 at 3:50 AM, Jack Morgenstein wrote: > On Monday 21 July 2008 19:46, Hal Rosenstock wrote: >> Out of curiousity, why is a module parameter needed to enable/disable >> QoS ? Is there a downside to enabling it in terms of the HCA ? >> >> -- Hal >> > > We noticed (after ofed 1.3 was released) that setting the QoS bit in the HCA > resulted in using more HW resources. Would you elaborate on which ones ? Thanks. -- Hal > Since Qos is usually not used we decided to disable the feature by default, > and allow enabling it when it is needed. > > In ofed 1.3.1 the default value was therefore changed to be OFF. > > - Jack > From hal.rosenstock at gmail.com Sun Jul 27 04:43:07 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 27 Jul 2008 07:43:07 -0400 Subject: [ofa-general] [PATCH] opensm/osm_inform.c: Fix compile warning Message-ID: opensm/osm_inform.c: Fix compile warning osm_inform.c: In function `__osm_send_report': osm_inform.c:319: warning: `status' might be used uninitialized in this function Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index c8c06b2..ca972a1 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -316,7 +316,7 @@ static ib_api_status_t __osm_send_report(IN osm_infr_t * p_i ib_mad_t *p_mad; ib_sa_mad_t *p_sa_mad; static atomic32_t trap_fwd_trans_id = 0x02DAB000; - ib_api_status_t status; + ib_api_status_t status = IB_SUCCESS; osm_log_t *p_log = p_infr_rec->sa->p_log; OSM_LOG_ENTER(p_log); From eli at mellanox.co.il Sun Jul 27 05:13:59 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 27 Jul 2008 15:13:59 +0300 Subject: [ofa-general] [PATCH] IB/core: return immediate error when registering zero length buffer Message-ID: <20080727121359.GA8941@mtls03> Registering a zero length buffer is meaningless so check this at ib_umem_get() and return immediately with error if that is the case. If we don't do this, we get an inconsistent behaviour. For example, if we register a buffer whose start address is not page aligned, we will end up registering one page, while if the address is page aligned, we will have zero pages -- the reason for this comes from the following statement: npages = PAGE_ALIGN(size + umem->offset) >> PAGE_SHIFT; Problem found by: Ronni Zimmermann Signed-off-by: Eli Cohen --- drivers/infiniband/core/umem.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 6f7c096..2541ef8 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -89,6 +89,9 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr, int i; DEFINE_DMA_ATTRS(attrs); + if (!size) + return ERR_PTR(-ENOMEM); + if (dmasync) dma_set_attr(DMA_ATTR_WRITE_BARRIER, &attrs); -- 1.5.6 From ronli at voltaire.com Sun Jul 27 08:23:14 2008 From: ronli at voltaire.com (Ron Livne) Date: Sun, 27 Jul 2008 15:23:14 +0000 (UTC) Subject: [ofa-general] [PATCH 3/3 v3] ib/uverbs: add support for create_qp_expanded in uverbs Message-ID: This patch adds support for create_qp_expanded to the uverbs. It uses the reserved bitmap in ib_uverbs_create_qp to transfer the new creation flags from the user space to the kernel. Changes in v2: Minimized code duplication by adding the function ib_uverbs_create_qp_common. LSO now can not be activated through user space. Changes in v3: Added compatibility for old libibverbs. Added field __u32 create_flags to struct ib_uverbs_create_qp_expanded. Deleted the function ib_uverbs_create_qp_common from v2. Signed-off-by: Ron Livne uevent.uobject, cmd.user_handle, file->ucontext, &qp_lock_key); + down_write(&obj->uevent.uobject.mutex); + + srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? + idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; + xrcd = cmd.qp_type == IB_QPT_XRC ? + idr_read_xrcd(cmd.srq_handle, file->ucontext, &xrcd_uobj) : NULL; + pd = idr_read_pd(cmd.pd_handle, file->ucontext); + scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); + rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? + scq : idr_read_cq(cmd.recv_cq_handle, file->ucontext, 1); + + if (!pd || !scq || !rcq || (cmd.is_srq && !srq) || + (cmd.qp_type == IB_QPT_XRC && !xrcd)) { + ret = -EINVAL; + goto err_put; + } + + attr.event_handler = ib_uverbs_qp_event_handler; + attr.qp_context = file; + attr.send_cq = scq; + attr.recv_cq = rcq; + attr.srq = srq; + attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; + attr.qp_type = cmd.qp_type; + attr.xrc_domain = xrcd; + attr.create_flags = 0; + + attr.cap.max_send_wr = cmd.max_send_wr; + attr.cap.max_recv_wr = cmd.max_recv_wr; + attr.cap.max_send_sge = cmd.max_send_sge; + attr.cap.max_recv_sge = cmd.max_recv_sge; + attr.cap.max_inline_data = cmd.max_inline_data; + + obj->uevent.events_reported = 0; + INIT_LIST_HEAD(&obj->uevent.event_list); + INIT_LIST_HEAD(&obj->mcast_list); + + qp = pd->device->create_qp(pd, &attr, &udata); + if (IS_ERR(qp)) { + ret = PTR_ERR(qp); + goto err_put; + } + + qp->device = pd->device; + qp->pd = pd; + qp->send_cq = attr.send_cq; + qp->recv_cq = attr.recv_cq; + qp->srq = attr.srq; + qp->uobject = &obj->uevent.uobject; + qp->event_handler = attr.event_handler; + qp->qp_context = attr.qp_context; + qp->qp_type = attr.qp_type; + qp->xrcd = attr.xrc_domain; + atomic_inc(&pd->usecnt); + atomic_inc(&attr.send_cq->usecnt); + atomic_inc(&attr.recv_cq->usecnt); + if (attr.srq) + atomic_inc(&attr.srq->usecnt); + else if (attr.xrc_domain) + atomic_inc(&attr.xrc_domain->usecnt); + + obj->uevent.uobject.object = qp; + ret = idr_add_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); + if (ret) + goto err_destroy; + + memset(&resp, 0, sizeof resp); + resp.qpn = qp->qp_num; + resp.qp_handle = obj->uevent.uobject.id; + resp.max_recv_sge = attr.cap.max_recv_sge; + resp.max_send_sge = attr.cap.max_send_sge; + resp.max_recv_wr = attr.cap.max_recv_wr; + resp.max_send_wr = attr.cap.max_send_wr; + resp.max_inline_data = attr.cap.max_inline_data; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + ret = -EFAULT; + goto err_copy; + } + + put_pd_read(pd); + put_cq_read(scq); + if (rcq != scq) + put_cq_read(rcq); + if (srq) + put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd_uobj); + + mutex_lock(&file->mutex); + list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); + mutex_unlock(&file->mutex); + + obj->uevent.uobject.live = 1; + + up_write(&obj->uevent.uobject.mutex); + + return in_len; + +err_copy: + idr_remove_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); + +err_destroy: + ib_destroy_qp(qp); + +err_put: + if (pd) + put_pd_read(pd); + if (scq) + put_cq_read(scq); + if (rcq && rcq != scq) + put_cq_read(rcq); + if (srq) + put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd_uobj); + + put_uobj_write(&obj->uevent.uobject); + return ret; +} + +ssize_t ib_uverbs_create_qp_expanded(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) { - struct ib_uverbs_create_qp cmd; + struct ib_uverbs_create_qp_expanded cmd; struct ib_uverbs_create_qp_resp resp; struct ib_udata udata; struct ib_uqp_object *obj; @@ -1078,7 +1232,6 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, goto err_put; } - attr.create_flags = 0; attr.event_handler = ib_uverbs_qp_event_handler; attr.qp_context = file; attr.send_cq = scq; @@ -1087,7 +1240,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; attr.xrc_domain = xrcd; - attr.create_flags = 0; + attr.create_flags = cmd.create_flags; attr.cap.max_send_wr = cmd.max_send_wr; attr.cap.max_recv_wr = cmd.max_recv_wr; diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 1a96c35..cb435be 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -117,6 +117,7 @@ static ssize_t (*uverbs_cmd_table[])(struct ib_uverbs_file *file, [IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP] = ib_uverbs_query_xrc_rcv_qp, [IB_USER_VERBS_CMD_REG_XRC_RCV_QP] = ib_uverbs_reg_xrc_rcv_qp, [IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP] = ib_uverbs_unreg_xrc_rcv_qp, + [IB_USER_VERBS_CMD_CREATE_QP_EXPANDED] = ib_uverbs_create_qp_expanded, }; static struct vfsmount *uverbs_event_mnt; diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 030f696..f954533 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -728,7 +728,8 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) (1ull << IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP) | (1ull << IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP) | (1ull << IB_USER_VERBS_CMD_REG_XRC_RCV_QP) | - (1ull << IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP); + (1ull << IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP) | + (1ull << IB_USER_VERBS_CMD_CREATE_QP_EXPANDED); } if (init_node_data(ibdev)) diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 0d3f770..c65f88b 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -518,9 +518,6 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } else { qp->sq_no_prefetch = 0; - if (init_attr->create_flags & IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) - qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; - if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) qp->flags |= MLX4_IB_QP_LSO; @@ -559,6 +556,10 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } } + if (init_attr->create_flags & + IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) + qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; + err = mlx4_qp_alloc(dev->dev, sqpn, &qp->mqp); if (err) goto err_wrid; @@ -705,8 +706,11 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)) return ERR_PTR(-EINVAL); - if (init_attr->create_flags && - (pd->uobject || init_attr->qp_type != IB_QPT_UD)) + if (init_attr->create_flags && init_attr->qp_type != IB_QPT_UD) + return ERR_PTR(-EINVAL); + + if ((init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) && + pd->uobject) return ERR_PTR(-EINVAL); switch (init_attr->qp_type) { diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h index 0df90d8..300474f 100644 --- a/include/rdma/ib_user_verbs.h +++ b/include/rdma/ib_user_verbs.h @@ -90,6 +90,7 @@ enum { IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, IB_USER_VERBS_CMD_REG_XRC_RCV_QP, IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED, }; /* @@ -411,6 +412,27 @@ struct ib_uverbs_create_qp { __u64 driver_data[0]; }; +struct ib_uverbs_create_qp_expanded { + __u64 response; + __u64 user_handle; + __u32 pd_handle; + __u32 send_cq_handle; + __u32 recv_cq_handle; + __u32 srq_handle; + __u32 max_send_wr; + __u32 max_recv_wr; + __u32 max_send_sge; + __u32 max_recv_sge; + __u32 max_inline_data; + __u8 sq_sig_all; + __u8 qp_type; + __u8 is_srq; + __u8 reserved; + __u32 reserved1; + __u32 create_flags; + __u64 driver_data[0]; +}; + struct ib_uverbs_create_qp_resp { __u32 qp_handle; __u32 qpn; From ronli.voltaire at gmail.com Sun Jul 27 05:20:49 2008 From: ronli.voltaire at gmail.com (Ron Livne) Date: Sun, 27 Jul 2008 15:20:49 +0300 Subject: [ofa-general] [PATCH 3/3 v3] ib/uverbs: add support for create_qp_expanded in uverbs In-Reply-To: References: Message-ID: <3b5e77ad0807270520s2217eee7i9991c9e041063fd8@mail.gmail.com> Jack, I know there is some duplicated code in ib_uverbs_create_qp and ib_uverbs_create_qp_expanded. However, only a small part of this code can be extracted to a common function. I think it'll be ugly to extract it. Do you think I should extract it anyway to avoid duplications? Thank you, Ron On Sun, Jul 27, 2008 at 6:23 PM, Ron Livne wrote: > This patch adds support for create_qp_expanded > to the uverbs. > It uses the reserved bitmap in ib_uverbs_create_qp > to transfer the new creation flags from the user space > to the kernel. > > Changes in v2: > Minimized code duplication by adding the function > ib_uverbs_create_qp_common. > > LSO now can not be activated through user space. > > Changes in v3: > Added compatibility for old libibverbs. > Added field __u32 create_flags to struct ib_uverbs_create_qp_expanded. > Deleted the function ib_uverbs_create_qp_common from v2. > > Signed-off-by: Ron Livne > diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h > index b55f0d7..ae9f9a8 100644 > --- a/drivers/infiniband/core/uverbs.h > +++ b/drivers/infiniband/core/uverbs.h > @@ -214,6 +214,7 @@ IB_UVERBS_DECLARE_CMD(modify_xrc_rcv_qp); > IB_UVERBS_DECLARE_CMD(query_xrc_rcv_qp); > IB_UVERBS_DECLARE_CMD(reg_xrc_rcv_qp); > IB_UVERBS_DECLARE_CMD(unreg_xrc_rcv_qp); > +IB_UVERBS_DECLARE_CMD(create_qp_expanded); > > > #endif /* UVERBS_H */ > diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c > index 4402a07..a9c1485 100644 > --- a/drivers/infiniband/core/uverbs_cmd.c > +++ b/drivers/infiniband/core/uverbs_cmd.c > @@ -1030,10 +1030,164 @@ ssize_t ib_uverbs_destroy_cq(struct ib_uverbs_file *file, > } > > ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, > + const char __user *buf, int in_len, > + int out_len) > +{ > + struct ib_uverbs_create_qp cmd; > + struct ib_uverbs_create_qp_resp resp; > + struct ib_udata udata; > + struct ib_uqp_object *obj; > + struct ib_pd *pd; > + struct ib_cq *scq, *rcq; > + struct ib_srq *srq; > + struct ib_qp *qp; > + struct ib_qp_init_attr attr; > + struct ib_xrcd *xrcd; > + struct ib_uobject *xrcd_uobj; > + int ret; > + > + if (out_len < sizeof resp) > + return -ENOSPC; > + > + if (copy_from_user(&cmd, buf, sizeof cmd)) > + return -EFAULT; > + > + INIT_UDATA(&udata, buf + sizeof cmd, > + (unsigned long) cmd.response + sizeof resp, > + in_len - sizeof cmd, out_len - sizeof resp); > + > + obj = kmalloc(sizeof *obj, GFP_KERNEL); > + if (!obj) > + return -ENOMEM; > + > + init_uobj(&obj->uevent.uobject, cmd.user_handle, file->ucontext, &qp_lock_key); > + down_write(&obj->uevent.uobject.mutex); > + > + srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? > + idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; > + xrcd = cmd.qp_type == IB_QPT_XRC ? > + idr_read_xrcd(cmd.srq_handle, file->ucontext, &xrcd_uobj) : NULL; > + pd = idr_read_pd(cmd.pd_handle, file->ucontext); > + scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); > + rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? > + scq : idr_read_cq(cmd.recv_cq_handle, file->ucontext, 1); > + > + if (!pd || !scq || !rcq || (cmd.is_srq && !srq) || > + (cmd.qp_type == IB_QPT_XRC && !xrcd)) { > + ret = -EINVAL; > + goto err_put; > + } > + > + attr.event_handler = ib_uverbs_qp_event_handler; > + attr.qp_context = file; > + attr.send_cq = scq; > + attr.recv_cq = rcq; > + attr.srq = srq; > + attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; > + attr.qp_type = cmd.qp_type; > + attr.xrc_domain = xrcd; > + attr.create_flags = 0; > + > + attr.cap.max_send_wr = cmd.max_send_wr; > + attr.cap.max_recv_wr = cmd.max_recv_wr; > + attr.cap.max_send_sge = cmd.max_send_sge; > + attr.cap.max_recv_sge = cmd.max_recv_sge; > + attr.cap.max_inline_data = cmd.max_inline_data; > + > + obj->uevent.events_reported = 0; > + INIT_LIST_HEAD(&obj->uevent.event_list); > + INIT_LIST_HEAD(&obj->mcast_list); > + > + qp = pd->device->create_qp(pd, &attr, &udata); > + if (IS_ERR(qp)) { > + ret = PTR_ERR(qp); > + goto err_put; > + } > + > + qp->device = pd->device; > + qp->pd = pd; > + qp->send_cq = attr.send_cq; > + qp->recv_cq = attr.recv_cq; > + qp->srq = attr.srq; > + qp->uobject = &obj->uevent.uobject; > + qp->event_handler = attr.event_handler; > + qp->qp_context = attr.qp_context; > + qp->qp_type = attr.qp_type; > + qp->xrcd = attr.xrc_domain; > + atomic_inc(&pd->usecnt); > + atomic_inc(&attr.send_cq->usecnt); > + atomic_inc(&attr.recv_cq->usecnt); > + if (attr.srq) > + atomic_inc(&attr.srq->usecnt); > + else if (attr.xrc_domain) > + atomic_inc(&attr.xrc_domain->usecnt); > + > + obj->uevent.uobject.object = qp; > + ret = idr_add_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); > + if (ret) > + goto err_destroy; > + > + memset(&resp, 0, sizeof resp); > + resp.qpn = qp->qp_num; > + resp.qp_handle = obj->uevent.uobject.id; > + resp.max_recv_sge = attr.cap.max_recv_sge; > + resp.max_send_sge = attr.cap.max_send_sge; > + resp.max_recv_wr = attr.cap.max_recv_wr; > + resp.max_send_wr = attr.cap.max_send_wr; > + resp.max_inline_data = attr.cap.max_inline_data; > + > + if (copy_to_user((void __user *) (unsigned long) cmd.response, > + &resp, sizeof resp)) { > + ret = -EFAULT; > + goto err_copy; > + } > + > + put_pd_read(pd); > + put_cq_read(scq); > + if (rcq != scq) > + put_cq_read(rcq); > + if (srq) > + put_srq_read(srq); > + if (xrcd) > + put_xrcd_read(xrcd_uobj); > + > + mutex_lock(&file->mutex); > + list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); > + mutex_unlock(&file->mutex); > + > + obj->uevent.uobject.live = 1; > + > + up_write(&obj->uevent.uobject.mutex); > + > + return in_len; > + > +err_copy: > + idr_remove_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); > + > +err_destroy: > + ib_destroy_qp(qp); > + > +err_put: > + if (pd) > + put_pd_read(pd); > + if (scq) > + put_cq_read(scq); > + if (rcq && rcq != scq) > + put_cq_read(rcq); > + if (srq) > + put_srq_read(srq); > + if (xrcd) > + put_xrcd_read(xrcd_uobj); > + > + put_uobj_write(&obj->uevent.uobject); > + return ret; > +} > + > +ssize_t ib_uverbs_create_qp_expanded(struct ib_uverbs_file *file, > const char __user *buf, int in_len, > int out_len) > { > - struct ib_uverbs_create_qp cmd; > + struct ib_uverbs_create_qp_expanded cmd; > struct ib_uverbs_create_qp_resp resp; > struct ib_udata udata; > struct ib_uqp_object *obj; > @@ -1078,7 +1232,6 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, > goto err_put; > } > > - attr.create_flags = 0; > attr.event_handler = ib_uverbs_qp_event_handler; > attr.qp_context = file; > attr.send_cq = scq; > @@ -1087,7 +1240,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, > attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; > attr.qp_type = cmd.qp_type; > attr.xrc_domain = xrcd; > - attr.create_flags = 0; > + attr.create_flags = cmd.create_flags; > > attr.cap.max_send_wr = cmd.max_send_wr; > attr.cap.max_recv_wr = cmd.max_recv_wr; > diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c > index 1a96c35..cb435be 100644 > --- a/drivers/infiniband/core/uverbs_main.c > +++ b/drivers/infiniband/core/uverbs_main.c > @@ -117,6 +117,7 @@ static ssize_t (*uverbs_cmd_table[])(struct ib_uverbs_file *file, > [IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP] = ib_uverbs_query_xrc_rcv_qp, > [IB_USER_VERBS_CMD_REG_XRC_RCV_QP] = ib_uverbs_reg_xrc_rcv_qp, > [IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP] = ib_uverbs_unreg_xrc_rcv_qp, > + [IB_USER_VERBS_CMD_CREATE_QP_EXPANDED] = ib_uverbs_create_qp_expanded, > }; > > static struct vfsmount *uverbs_event_mnt; > diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c > index 030f696..f954533 100644 > --- a/drivers/infiniband/hw/mlx4/main.c > +++ b/drivers/infiniband/hw/mlx4/main.c > @@ -728,7 +728,8 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) > (1ull << IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP) | > (1ull << IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP) | > (1ull << IB_USER_VERBS_CMD_REG_XRC_RCV_QP) | > - (1ull << IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP); > + (1ull << IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP) | > + (1ull << IB_USER_VERBS_CMD_CREATE_QP_EXPANDED); > } > > if (init_node_data(ibdev)) > diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c > index 0d3f770..c65f88b 100644 > --- a/drivers/infiniband/hw/mlx4/qp.c > +++ b/drivers/infiniband/hw/mlx4/qp.c > @@ -518,9 +518,6 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, > } else { > qp->sq_no_prefetch = 0; > > - if (init_attr->create_flags & IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) > - qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; > - > if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) > qp->flags |= MLX4_IB_QP_LSO; > > @@ -559,6 +556,10 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, > } > } > > + if (init_attr->create_flags & > + IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) > + qp->flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK; > + > err = mlx4_qp_alloc(dev->dev, sqpn, &qp->mqp); > if (err) > goto err_wrid; > @@ -705,8 +706,11 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, > IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)) > return ERR_PTR(-EINVAL); > > - if (init_attr->create_flags && > - (pd->uobject || init_attr->qp_type != IB_QPT_UD)) > + if (init_attr->create_flags && init_attr->qp_type != IB_QPT_UD) > + return ERR_PTR(-EINVAL); > + > + if ((init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) && > + pd->uobject) > return ERR_PTR(-EINVAL); > > switch (init_attr->qp_type) { > diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h > index 0df90d8..300474f 100644 > --- a/include/rdma/ib_user_verbs.h > +++ b/include/rdma/ib_user_verbs.h > @@ -90,6 +90,7 @@ enum { > IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP, > IB_USER_VERBS_CMD_REG_XRC_RCV_QP, > IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP, > + IB_USER_VERBS_CMD_CREATE_QP_EXPANDED, > }; > > /* > @@ -411,6 +412,27 @@ struct ib_uverbs_create_qp { > __u64 driver_data[0]; > }; > > +struct ib_uverbs_create_qp_expanded { > + __u64 response; > + __u64 user_handle; > + __u32 pd_handle; > + __u32 send_cq_handle; > + __u32 recv_cq_handle; > + __u32 srq_handle; > + __u32 max_send_wr; > + __u32 max_recv_wr; > + __u32 max_send_sge; > + __u32 max_recv_sge; > + __u32 max_inline_data; > + __u8 sq_sig_all; > + __u8 qp_type; > + __u8 is_srq; > + __u8 reserved; > + __u32 reserved1; > + __u32 create_flags; > + __u64 driver_data[0]; > +}; > + > struct ib_uverbs_create_qp_resp { > __u32 qp_handle; > __u32 qpn; > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From sashak at voltaire.com Sun Jul 27 05:55:49 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 15:55:49 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_inform.c: Fix compile warning In-Reply-To: References: Message-ID: <20080727125549.GS5067@sashak.voltaire.com> On 07:43 Sun 27 Jul , Hal Rosenstock wrote: > opensm/osm_inform.c: Fix compile warning > > osm_inform.c: In function `__osm_send_report': > osm_inform.c:319: warning: `status' might be used uninitialized in this function > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From tziporet at dev.mellanox.co.il Sun Jul 27 06:07:33 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 27 Jul 2008 16:07:33 +0300 Subject: [ofa-general] Terrible TCP Qperf Performance with Centos 5.2 and default settings In-Reply-To: References: Message-ID: <488C7315.2060309@mellanox.co.il> Robert Dunkley wrote: > > I see very low qperf TCP performance of IPOIB on Centos 5.2 by default > (RDMA performance is fine). > Can you use netpef for IPoIB testing - this will be able to compare to measurements we take here > > Centos sets the maximum MTU for ib0 on boot automatically. For CM > centos sets the maximum MTU that is just slightly less than the > logical 65536 setting. Connected mode only seems to like nice round > MTU values (eg. 32768 or 2048) so the default setting always gives > terrible performance. > In OFED 1.3.1 we have a problem when moving from UD to CM in setting of the MTU of CM mode to the maximum allowable value. To workaround this feature, the user requires to do the following actions to ensure the MTU setting is done correctly: 1. ifconfig ib0 2. Take note of the MTU specified, let's call it xmtu. Let's call ymtu = xmtu - 10 3. ifconfig ib0 mtu ymtu 4. ifconfig ib0 mtu xmtu The above procedure is valid for any ib interface. Tziporet From sashak at voltaire.com Sun Jul 27 06:09:16 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 16:09:16 +0300 Subject: [ofa-general] [PATCH] opensm: remove -rpath from LDFLAGS Message-ID: <20080727130916.GT5067@sashak.voltaire.com> From: Doug Ledford Fedora is very picky that things pass rpmlint before they are allowed into the distribution. They only allow things that rpmlint flags as an error in the case of obvious necessity. One of the things that came up in the review process is that opensm still uses rpath settings in its various Makefile.am files. The use of rpath at all is considered to be an error and is currently blocking things. I went in to the Makefile.am files and attempted to clean them up. Signed-off-by: Sasha Khapyorsky --- opensm/config/osmvsel.m4 | 1 - opensm/opensm/Makefile.am | 6 +++--- opensm/osmtest/Makefile.am | 5 ++--- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/opensm/config/osmvsel.m4 b/opensm/config/osmvsel.m4 index 2999881..7e68813 100644 --- a/opensm/config/osmvsel.m4 +++ b/opensm/config/osmvsel.m4 @@ -68,7 +68,6 @@ if test $with_osmv = "openib"; then OSMV_CFLAGS="-DOSM_VENDOR_INTF_OPENIB" OSMV_INCLUDES="-I\$(srcdir)/../include -I\$(srcdir)/../../libibcommon/include -I\$(srcdir)/../../libibumad/include -I\$(includedir)" OSMV_LDADD="-L\$(abs_srcdir)/../../libibumad/.libs -L\$(abs_srcdir)/../../libibcommon/.libs -L\$(libdir) -libumad -libcommon" - OSMV_LDADD="-Wl,--rpath -Wl,\$(abs_srcdir)/../../libibumad/.libs -Wl,--rpath -Wl,\$(abs_srcdir)/../../libibcommon/.libs -Wl,--rpath -Wl,\$(libdir) $OSMV_LDADD" if test "x$with_umad_libs" != "x"; then OSMV_LDADD="-L$with_umad_libs $OSMV_LDADD" diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 80f5a01..c8a9980 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -79,7 +79,7 @@ endif if OSMV_SIM USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ libopensm.la: $(libopensm_la_OBJECTS) $(libopensm_la_DEPENDENCIES) - $(USEGPPLINK) -rpath $(libdir) $(libopensm_la_LDFLAGS) $(libopensm_la_OBJECTS) $(libopensm_la_LIBADD) $(LIBS) + $(USEGPPLINK) $(libopensm_la_LDFLAGS) $(libopensm_la_OBJECTS) $(libopensm_la_LIBADD) $(LIBS) opensm$(EXEEXT): $(opensm_OBJECTS) $(opensm_DEPENDENCIES) @rm -f opensm$(EXEEXT) $(USEGPPLINK) $(opensm_LDFLAGS) $(opensm_OBJECTS) $(opensm_LDADD) $(LIBS) @@ -87,9 +87,9 @@ endif # we need to be able to load libraries from local build subtree before make install # we always give precedence to local tree libs and then use the pre-installed ones. -opensm_LDADD = -L../complib -L../libvendor -L. $(OSMV_LDADD) -lopensm -losmcomp -losmvendor +opensm_LDADD = -L../complib -losmcomp -L../libvendor -losmvendor -L. -lopensm $(OSMV_LDADD) -opensm_LDFLAGS = -Wl,--rpath -Wl,$(libdir) -lpthread -ldl +opensm_LDFLAGS = -lpthread -ldl opensmincludedir = $(includedir)/infiniband/opensm diff --git a/opensm/osmtest/Makefile.am b/opensm/osmtest/Makefile.am index cb1c755..c9b6019 100644 --- a/opensm/osmtest/Makefile.am +++ b/opensm/osmtest/Makefile.am @@ -18,10 +18,9 @@ osmtest_CFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $( else osmtest_CFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT $(DBGFLAGS) endif -osmtest_LDADD = -L../complib -L../libvendor -L../opensm -L$(libdir) -L. \ - $(OSMV_LDADD) -lopensm -losmcomp -losmvendor +osmtest_LDADD = -L../complib -losmcomp -L../libvendor -losmvendor -L../opensm -lopensm $(OSMV_LDADD) -osmtest_LDFLAGS = -Wl,--rpath -Wl,$(libdir) -lpthread -L../opensm +osmtest_LDFLAGS = -lpthread # for linking with the simulator client library we have to use g++: if OSMV_SIM USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ -- 1.5.5.1.178.g1f811 From jackm at dev.mellanox.co.il Sun Jul 27 06:56:57 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 27 Jul 2008 16:56:57 +0300 Subject: [ofa-general] [PATCH 3/3 v3] ib/uverbs: add support for create_qp_expanded in uverbs In-Reply-To: <3b5e77ad0807270520s2217eee7i9991c9e041063fd8@mail.gmail.com> References: <3b5e77ad0807270520s2217eee7i9991c9e041063fd8@mail.gmail.com> Message-ID: <200807271656.58407.jackm@dev.mellanox.co.il> On Sunday 27 July 2008 15:20, Ron Livne wrote: > Do you think I should extract it anyway to avoid duplications? > No, it's not worth it. - Jack From 100joguio at ltbaroda.com Sun Jul 27 06:08:11 2008 From: 100joguio at ltbaroda.com (grannie tracy) Date: Sun, 27 Jul 2008 13:08:11 +0000 Subject: [ofa-general] Best of Bally, Coach, UGG Message-ID: <000601c8eff8$0248a4ae$d7ec52ad@bbgiler> The world's great luxury store for shoes and bags is just one click away. Recommended by tens thousands of grateful customers all around the world, we have dozens of famous brands including: Prada Bally Coach Dsquared Louis Vuitton Here you willc find thousands of best designs for shoes, and leather products, at at temp't1ng priceE. Sale close this weekend, so visit shop today and start pampering yourself and your loved ones. - Visit our site: www.minepint[DOT]com (copy this link then replace "[DOT]" to ".") From swise at opengridcomputing.com Sat Jul 26 08:58:08 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Sat, 26 Jul 2008 10:58:08 -0500 Subject: [ofa-general] Re: [ewg] OFED 1.4 alpha release is ready In-Reply-To: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> References: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> Message-ID: <488B4990.2010208@opengridcomputing.com> How can we claim an alpha release when the tree is still based on linux-2.6.26 and doesn't have a large chunk of the new features integrated? Tziporet Koren wrote: > Hi, > > OFED 1.4 alpha release is available on > http://www.openfabrics.org/downloads/OFED/ofed-1.4/OFED-1.4-alpha1.tgz > > To get BUILD_ID run ofed_info > > Please report any issues in bugzilla https://bugs.openfabrics.org/ for > OFED 1.4 > > The beta release is expected on August 7 2008 > > Tziporet & Vlad > > > ======================================================================== > > Release information: > -------------------- > Linux Operating Systems: > - RedHat EL4 up5: 2.6.9-55.ELsmp > - RedHat EL4 up6: 2.6.9-67.ELsmp > - RedHat EL5: 2.6.18-8.el5 > - RedHat EL5 up1: 2.6.18-53.el5 > - RedHat EL5 up2: 2.6.18-92.el5 > - Fedora C9: 2.6.25-14.fc9 * > - SLES10: 2.6.16.21-0.8-smp > - SLES10 SP1: 2.6.16.46-0.12-smp > - SLES10 SP1 up1: 2.6.16.53-0.16-smp > - SLES10 SP2: 2.6.16.60-0.21-smp > - OpenSuSE 10.3: 2.6.22.5-31 * > - kernel.org: 2.6.26 > > * OSes that are partially tested > > Systems: > * x86_64 > * x86 > * ia64 > * ppc64 > > > Main Changes from OFED 1.3 > ========================== > 1. General changes > o Kernel code based on 2.6.26 (will be 2.6.27) > o Added iSER target package > o Added NFS-RDMA support (for 2.6.26 only for now) > 2. IPoIB > o LRO support - when we move to 2.6.27 > 3. SDP > o Bug fixes in the state machine and close flow > 4. qlgc_vnic > o Support for hotswap of EVIC and dynamic update of existing > connections > with the addition of QLogic dynamic update daemon. > o Performance improvements in handling of Ethernet > broadcast/multicast > traffic. > 5. RDS (for now only kernel 2.6.26 is working) > o GA of RDMA API (using FMRs) - RDS API version 3 > o iWARP support > 6. uDAPL > o Added socket based CM - for both scalability and interop with > Windows > o Added UD extensions - for version 2.0 only > o v1 library package has been renamed to compat-dapl-1.2.8-1 > 7. Management > o OpenSM > - APM - disjoint paths > - Path balancing for LMC + console diagnostics > - OpenSM configuration unification > - MGID to MLID mapping for IPv6 SNM > - Routing engines chain > - IBA 1.2.1 additions > - Failover/Handover improvements > o ibutils: > - Congestion Control > - Report created in CSV format > o Diagnostic tools: > - ibnetdiscover library - to accelerate another tools > 8. Low level drivers: > o mlx4: Enable Virtual Protocol Interconnect - Eth and IB on the > same device > > > Tasks that should be completed for the beta: > ============================================ > 1. Move to kernel base 2.6.27-rc1 with the features: > - New verbs to support BMME (Fast memory thru send queue, Local > invalidate send work requests, > Read with invalidate. > - IPoIB LRO > 2. RDS to work on distro OSes > 3. NFS-RDMA to work on distro OSes > 4. New MPI versions: MVAPICH 1.1, OpenMPI 1.3, MVAPICH2 1.2 > 5. OSM: Cashed routing > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > _______________________________________________ ewg mailing list ewg at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg From sashak at voltaire.com Sun Jul 27 11:47:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 27 Jul 2008 21:47:05 +0300 Subject: [ofa-general] ***SPAM*** [PATCH] opensm/*/Makefile.am: remove explicit -lpthread and -ldl flags from Makefile.am In-Reply-To: <20080727130916.GT5067@sashak.voltaire.com> References: <20080727130916.GT5067@sashak.voltaire.com> Message-ID: <20080727184705.GV5067@sashak.voltaire.com> -lpthread and -ldl are standard things and can be generated by ./configure into LIBS variable and we don't need to specify this explicitly in Makefiles. If so remove already not used lopensm_LDFLAGS and osmtest_LDFLAGS variables too. Signed-off-by: Sasha Khapyorsky --- opensm/configure.in | 2 ++ opensm/opensm/Makefile.am | 4 +--- opensm/osmtest/Makefile.am | 1 - 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/opensm/configure.in b/opensm/configure.in index 7d04c14..7da932b 100644 --- a/opensm/configure.in +++ b/opensm/configure.in @@ -38,6 +38,8 @@ fi dnl Checks for libraries AC_CHECK_LIB(pthread, pthread_mutex_init, [], AC_MSG_ERROR([pthread_mutex_init() not found. libosmcomp requires libpthread.])) +AC_CHECK_LIB(dl, dlopen, [], + AC_MSG_ERROR([dlopen() not found. OpenSM requires libdl.])) dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index c8a9980..d938b27 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -82,15 +82,13 @@ libopensm.la: $(libopensm_la_OBJECTS) $(libopensm_la_DEPENDENCIES) $(USEGPPLINK) $(libopensm_la_LDFLAGS) $(libopensm_la_OBJECTS) $(libopensm_la_LIBADD) $(LIBS) opensm$(EXEEXT): $(opensm_OBJECTS) $(opensm_DEPENDENCIES) @rm -f opensm$(EXEEXT) - $(USEGPPLINK) $(opensm_LDFLAGS) $(opensm_OBJECTS) $(opensm_LDADD) $(LIBS) + $(USEGPPLINK) $(opensm_OBJECTS) $(opensm_LDADD) $(LIBS) endif # we need to be able to load libraries from local build subtree before make install # we always give precedence to local tree libs and then use the pre-installed ones. opensm_LDADD = -L../complib -losmcomp -L../libvendor -losmvendor -L. -lopensm $(OSMV_LDADD) -opensm_LDFLAGS = -lpthread -ldl - opensmincludedir = $(includedir)/infiniband/opensm opensminclude_HEADERS = $(srcdir)/../include/opensm/osm_base.h \ diff --git a/opensm/osmtest/Makefile.am b/opensm/osmtest/Makefile.am index c9b6019..97326a5 100644 --- a/opensm/osmtest/Makefile.am +++ b/opensm/osmtest/Makefile.am @@ -20,7 +20,6 @@ osmtest_CFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT $(DBGFLAGS) endif osmtest_LDADD = -L../complib -losmcomp -L../libvendor -losmvendor -L../opensm -lopensm $(OSMV_LDADD) -osmtest_LDFLAGS = -lpthread # for linking with the simulator client library we have to use g++: if OSMV_SIM USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ -- 1.5.5.1.178.g1f811 From swise at opengridcomputing.com Sun Jul 27 14:16:11 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Sun, 27 Jul 2008 16:16:11 -0500 Subject: [ofa-general] Re: [ewg] OFED 1.4 alpha release is ready In-Reply-To: <488C5AF5.1070002@mellanox.co.il> References: <5D49E7A8952DC44FB38C38FA0D758EAD0FE653@mtlexch01.mtl.com> <488B4990.2010208@opengridcomputing.com> <488C5AF5.1070002@mellanox.co.il> Message-ID: <488CE59B.3060900@opengridcomputing.com> Tziporet Koren wrote: > Steve Wise wrote: >> How can we claim an alpha release when the tree is still based on >> linux-2.6.26 and doesn't have a large chunk of the new features >> integrated? >> > This was the process we decided to follow at Sonoma. > I know some features are not there yet, but many already in and > starting moving the release process forward is important even if not > all features are in. > Ok. Thanks for the clarification. Steve. From ogerlitz at voltaire.com Sun Jul 27 22:34:37 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 28 Jul 2008 08:34:37 +0300 Subject: [ofa-general] [PATCH] IB/core: return immediate error when registering zero length buffer In-Reply-To: <20080727121359.GA8941@mtls03> References: <20080727121359.GA8941@mtls03> Message-ID: <488D5A6D.1070205@voltaire.com> Eli Cohen wrote: > + if (!size) > + return ERR_PTR(-ENOMEM); doesn't EINVAL makes more sense here? Or. From ogerlitz at voltaire.com Sun Jul 27 23:23:28 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 28 Jul 2008 09:23:28 +0300 Subject: [ofa-general] Direct / "Crossover" IPOIB In-Reply-To: References: <488840EC.1020207@voltaire.com> Message-ID: <488D65E0.3090301@voltaire.com> Robert Dunkley wrote: > What kind of bandwidth do other users see with TCP on qperf? using qperf I get the following rc_bw 1.95 GB/s rc_bi_bw 3.9 GB/s tcp_bw 995 MB/s - datagram mode mtu 2044 tcp_bw 1.15 GB/s - connected mode mtu 65520 my systems have ConnectX DDR HCA (25418) and PCIe gen2 From your rc_bw results it seems that your systems have gen1 PCIe, and from the ibstat output I see that you have the one port Sinai HCA (25204). The datagram mode results with connectx should be better since it supports TCP stateless offloads such as checksum and LSO. To change the mode of ipoib use echo $mode > /sys/class/net/$dev/mode where $mode is connected or datagram As for latency, from my experience with qperf it doesn't give the optimized latency results, so for verbs based results I suggest to use ib_write_lat which is provided with the perftest package, for tcp latency results, you might want to use netperf with TCP_RR and UDP_RR tests, eg: > netperf -H 10.10.0.90 -D 1, -l 10 -fM -t TCP_RR -- -D L,R -m 4 -M 4 > netperf -H 10.10.0.90 -D 1, -l 10 -fM -t UDP_RR -- -m 4 -M 4 running these on my systems I see 25k (and above) transactions per second which suggests an RTT/2 latency of about 20us (40us / 2) you saw with qperf... Or From kaenmfwfyrgx at miyard.com Mon Jul 28 01:53:46 2008 From: kaenmfwfyrgx at miyard.com (esther) Date: Mon, 28 Jul 2008 00:53:46 -0800 Subject: [ofa-general] it`s esther Message-ID: Hi It`s esther again. Will you ever contact me? I made those nude pictures especially for you and I wont write to you again! If you wanna see them just drop me a line at: besther762 at wildpears.info From eli at dev.mellanox.co.il Mon Jul 28 01:09:53 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 28 Jul 2008 11:09:53 +0300 Subject: [ofa-general] [PATCH] IB/core: return immediate error when registering zero length buffer In-Reply-To: <488D5A6D.1070205@voltaire.com> References: <20080727121359.GA8941@mtls03> <488D5A6D.1070205@voltaire.com> Message-ID: <20080728080953.GA507@mtls03> On Mon, Jul 28, 2008 at 08:34:37AM +0300, Or Gerlitz wrote: > Eli Cohen wrote: >> + if (!size) >> + return ERR_PTR(-ENOMEM); > doesn't EINVAL makes more sense here? > Maybe, I'm not sure. From vlad at lists.openfabrics.org Mon Jul 28 02:58:10 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Mon, 28 Jul 2008 02:58:10 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080728-0200 daily build status Message-ID: <20080728095810.D8F21E609D6@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: From WileypotpourriGlass at samuel-beckett.net Mon Jul 28 03:03:19 2008 From: WileypotpourriGlass at samuel-beckett.net (Millard Strong) Date: Mon, 28 Jul 2008 10:03:19 +0000 Subject: [ofa-general] fate enzyme Message-ID: <1IX477EJXVWDA096@samuel-beckett.net> calamus atavism rerouted rocket rerouted structure? lamarck, bow lawsuit. vega calamus councilmen diversionary allergic frown, cutlass cutlass conjunct rocket sashay depressor. conjunct hodges. From olaf.kirch at oracle.com Mon Jul 28 05:25:31 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Mon, 28 Jul 2008 14:25:31 +0200 Subject: [ofa-general] Code drop for OFED 1.4 Message-ID: <200807281425.32641.olaf.kirch@oracle.com> Hi, I have one more code drop for RDS in OFED 1.4. It fixes the backport problems with RDS, which prevented it from working on anything but mainline. As usual, please pull from http://www.openfabrics.org/git/?p=~okir/ofed_1_4/linux-2.6.git code-drop-20080728 sysctl backport: fix CTL_UNNUMBERED Defining CTL_UNNUMBERED as 0 in older kernels tells the sysctl code not to register proc files for these controls. Instead, we use -2 instead. sysctl backport fixes The new sysctl backport header could lead to kernel oopses because of two mistakes I made. The first problem is that we have to allocate two ctl_table structs per patch entry because each is a null terminated array. The other is that this path must remain allocated for the lifetime of the registration - we must free these arrays at unregister time. This patch fixes both issues. RDS: temporarily revert sk_alloc change The sk_alloc change in rds_create, from commit f122cf9f788336c793823d14c16a8fe6803ea714, was premature. It broke compilation on older kernels :-( Revert this change, then clean up the code, then do it right. RDS: clean up old sk_alloc compat code There's some backward compat code that tries to make things work on kernels that still have a sock->sk_protinfo field (rather than overloading struct socket). I do hope that the whole KERNEL_HAS_PROTO_REGISTER mess can go away, but I will leave that to someone else. Instead, this patch restructures the code just a little to get rid of some KERNEL_HAS_PROTO_REGISTER ifdefs in rds_create. RDS: fix sock creation backport issue This patch should fix the use of namespaces with sk_alloc in rds_create for good. It's still somewhat ugly, but I didn't find a more elegant way to support this. RDS: Update module info This patch updates the driver version to 4.0 (to match OFED 1.4), as well as the driver release date. I also added myself as MODULE_AUTHOR, even though there probably won't be too many RDS patches from me after this one... Kudos to the people I worked with on this, I really appreciated your input, feedback and support. Enjoy! If you need to reach me in the future, please contact me at okir at lst.de or (starting august 1st) okir at suse.de. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From hal.rosenstock at gmail.com Mon Jul 28 06:59:49 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 09:59:49 -0400 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080724031356.GD11924@sashak.voltaire.com> References: <20080709190538.GO12551@sashak.voltaire.com> <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> <20080724023252.GB11924@sashak.voltaire.com> <20080724031356.GD11924@sashak.voltaire.com> Message-ID: On Wed, Jul 23, 2008 at 11:13 PM, Sasha Khapyorsky wrote: > On 23:04 Wed 23 Jul , Hal Rosenstock wrote: >> On Wed, Jul 23, 2008 at 10:32 PM, Sasha Khapyorsky wrote: >> > On 22:19 Wed 23 Jul , Hal Rosenstock wrote: >> >> >> >> Are you saying >> >> it's going to work with those changes if the APIs are used ? >> > >> > func(void *p) was changed to func(const void *p), why not? >> >> Not that one; we already discussed that. I thought there was some >> other change. Am I mistaken ? > > So what? new functions? How this will affect backward compatibility? New apps built (out of tree) using new APIs with this tar ball won't link against older ones with same version which will lead to confusion. > Anyway I see this issue as pretty minor. It is fixed now Is the version in libibmad.map updated now ? > and will appear in next tarball release. What is your "policy" for updating the various versions on an API update ? -- Hal > Sasha > From hal.rosenstock at gmail.com Mon Jul 28 07:07:20 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 10:07:20 -0400 Subject: [ofa-general] opensm/osm_sa.c: GID format Message-ID: Should the GID format in opensm/osm_sa.c also be updated to use inet_ntop/inet_pton rather than PRIx64 ? If so, should it also support the old format for input so old files can still be imported ? -- Hal From hal.rosenstock at gmail.com Mon Jul 28 06:54:07 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 09:54:07 -0400 Subject: [ofa-general] ***SPAM*** [PATCH] complib/cl_event_wheel.c: Fix some printf typos Message-ID: complib/cl_event_wheel.c: Fix some printf typos Signed-off-by: Hal Rosenstock --- diff --git a/opensm/complib/cl_event_wheel.c b/opensm/complib/cl_event_wheel.c index ea9db33..ca06882 100644 --- a/opensm/complib/cl_event_wheel.c +++ b/opensm/complib/cl_event_wheel.c @@ -482,7 +482,7 @@ void __cl_event_wheel_dump(IN cl_event_wheel_t * const p_eve nt_wheel) p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - printf("Event key:0x%" PRIx64 " Conetxt:%s NumRegs:%u\n", + printf("Event key:0x%" PRIx64 " Context:%s NumRegs:%u\n", p_event->key, (char *)p_event->context, p_event->num_regs); @@ -498,7 +498,7 @@ void __cl_event_wheel_dump(IN cl_event_wheel_t * const p_eve nt_wheel) p_event = PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, map_item); - printf("Event key:0x%" PRIx64 " Conetxt:%s NumRegs:%u\n", + printf("Event key:0x%" PRIx64 " Context:%s NumRegs:%u\n", p_event->key, (char *)p_event->context, p_event->num_regs); @@ -513,7 +513,7 @@ void __cl_event_wheel_dump(IN cl_event_wheel_t * const p_eve nt_wheel) void __test_event_aging(uint64_t key, void *context) { printf("*****************************************************\n"); - printf("Aged key: 0x%" PRIx64 " Conetxt:%s\n", key, (char *)context); + printf("Aged key: 0x%" PRIx64 " Context:%s\n", key, (char *)context); } int main() -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-comp1 Type: application/octet-stream Size: 1445 bytes Desc: not available URL: From hal.rosenstock at gmail.com Mon Jul 28 06:54:16 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 09:54:16 -0400 Subject: [ofa-general] ***SPAM*** [PATCH] opensm/osm_lid_mgr.c: Convert lid range prints to decimal Message-ID: opensm/osm_lid_mgr.c: Convert lid range prints to decimal Signed-off-by: Hal Rosenstock --- diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c index 9472065..daeea40 100644 --- a/opensm/opensm/osm_lid_mgr.c +++ b/opensm/opensm/osm_lid_mgr.c @@ -571,7 +571,7 @@ static int __osm_lid_mgr_init_sweep(IN osm_lid_mgr_t * const p_mgr) cl_qlist_insert_tail(&p_mgr->free_ranges, &p_range->item); OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "new free lid range [0x%x:0x%x]\n", + "new free lid range [%u:%u]\n", p_range->min_lid, p_range->max_lid); p_range = NULL; } @@ -599,7 +599,7 @@ AfterScanningLids: p_range->max_lid = p_mgr->p_subn->max_ucast_lid_ho; cl_qlist_insert_tail(&p_mgr->free_ranges, &p_range->item); OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "final free lid range [0x%x:0x%x]\n", + "final free lid range [%u:%u]\n", p_range->min_lid, p_range->max_lid); } @@ -809,7 +809,7 @@ __osm_lid_mgr_get_port_lid(IN osm_lid_mgr_t * const p_mgr, *p_max_lid = min_lid + num_lids - 1; OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, "0x%016" PRIx64 - " lid range:[0x%x-0x%x] is free\n", + " lid range:[%u-%u] is free\n", guid, *p_min_lid, *p_max_lid); goto NewLidSet; } else @@ -820,7 +820,7 @@ __osm_lid_mgr_get_port_lid(IN osm_lid_mgr_t * const p_mgr, } else OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, "0x%016" PRIx64 " existing lid range:" - "0x%x:0x%x] is not lmc aligned\n", + "%u:%u] is not lmc aligned\n", guid, min_lid, min_lid + num_lids - 1); } @@ -831,7 +831,7 @@ __osm_lid_mgr_get_port_lid(IN osm_lid_mgr_t * const p_mgr, __osm_lid_mgr_find_free_lid_range(p_mgr, num_lids, p_min_lid, p_max_lid); OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, - "0x%016" PRIx64 " assigned a new lid range:[0x%x-0x%x]\n", + "0x%016" PRIx64 " assigned a new lid range:[%u-%u]\n", guid, *p_min_lid, *p_max_lid); lid_changed = 1; -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-lidmgr1 Type: application/octet-stream Size: 1981 bytes Desc: not available URL: From hal.rosenstock at gmail.com Mon Jul 28 07:02:15 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 10:02:15 -0400 Subject: ***SPAM*** Re: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080724101113.76537e5e.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> <20080724101113.76537e5e.weiny2@llnl.gov> Message-ID: On Thu, Jul 24, 2008 at 1:11 PM, Ira Weiny wrote: > >From 1f810dd6ae9a73549cacdbb26f464e6132675592 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 22 Jul 2008 16:31:36 -0700 > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM > > > Signed-off-by: Ira K. Weiny > --- > opensm/include/iba/ib_types.h | 2 + > opensm/opensm/libopensm.map | 1 + > opensm/opensm/osm_helper.c | 44 +++++++++++++++++++++++++++++++++++++++++ > opensm/opensm/osm_inform.c | 3 +- > opensm/opensm/osm_trap_rcv.c | 10 +++++++- > 5 files changed, 57 insertions(+), 3 deletions(-) > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > index 09ec257..fffce00 100644 > --- a/opensm/include/iba/ib_types.h > +++ b/opensm/include/iba/ib_types.h > @@ -7219,6 +7219,8 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) > #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) > > +char * OSM_API ib_get_trap_str(uint16_t trap_num); > + > /****f* IBA Base: Types/ib_notice_is_generic > * NAME > * ib_notice_is_generic > diff --git a/opensm/opensm/libopensm.map b/opensm/opensm/libopensm.map > index 759a4e8..7cd2aba 100644 > --- a/opensm/opensm/libopensm.map > +++ b/opensm/opensm/libopensm.map > @@ -17,6 +17,7 @@ OPENSM_1.5 { Doesn't this also need updating as an API is being added ? -- Hal > ib_get_sm_method_str; > ib_get_sm_attr_str; > ib_get_sa_attr_str; > + ib_get_trap_str; > osm_dump_port_info; > osm_dump_portinfo_record; > osm_dump_guidinfo_record; > diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c > index 21ff51c..cc76d5c 100644 > --- a/opensm/opensm/osm_helper.c > +++ b/opensm/opensm/osm_helper.c > @@ -2338,3 +2338,47 @@ const char *osm_get_sm_mgr_state_str(IN uint16_t state) > __osm_sm_mgr_state_str[state] : > __osm_sm_mgr_state_str[ARR_SIZE(__osm_sm_mgr_state_str) - 1]; > } > + > +char * OSM_API > +ib_get_trap_str(uint16_t trap_num) > +{ > + switch(trap_num) > + { > + case 64: > + return ("GID in service"); > + case 65: > + return ("GID out of service"); > + case 66: > + return ("New mcast group created"); > + case 67: > + return ("Mcast group deleted"); > + case 68: > + return ("UnPath, Path no longer valid"); > + case 69: > + return ("RePath, Path recomputed"); > + case 128: > + return ("Link state change"); > + case 129: > + return ("Local Link integrity threshold reached"); > + case 130: > + return ("Excessive Buffer Overrun Threshold reached"); > + case 131: > + return ("Flow Control Update watchdog timer expired"); > + case 144: > + return ("CapabilityMask, NodeDescription, Link [Width|Speed] Enabled changed"); > + case 145: > + return ("System Image GUID changed"); > + case 256: > + return ("Bad M_Key"); > + case 257: > + return ("Bad P_Key"); > + case 258: > + return ("Bad Q_Key"); > + case 259: > + return ("Bad P_Key (switch external port)"); > + default: > + break; > + } > + return ("Unknown"); > +} > + > diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c > index e6146f7..ebc13bb 100644 > --- a/opensm/opensm/osm_inform.c > +++ b/opensm/opensm/osm_inform.c > @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, > /* an official Event information log */ > if (ib_notice_is_generic(p_ntc)) { > OSM_LOG(p_log, OSM_LOG_INFO, > - "Reporting Generic Notice type:%u num:%u" > + "Reporting Generic Notice type:%u num:%u (%s)" > " from LID:%u GID:0x%016" PRIx64 > ",0x%016" PRIx64 "\n", > ib_notice_get_type(p_ntc), > cl_ntoh16(p_ntc->g_or_v.generic.trap_num), > + ib_get_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), > cl_ntoh16(p_ntc->issuer_lid), > cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), > cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) > diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c > index ae7ab05..03cbd26 100644 > --- a/opensm/opensm/osm_trap_rcv.c > +++ b/opensm/opensm/osm_trap_rcv.c > @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > || (p_ntci->g_or_v.generic.trap_num == > CL_HTON16(131))) > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " > + "Received Generic Notice type:0x%02X " > + "num:%u (%s) Producer:%u (%s) " > "from LID:%u Port %d TID:0x%016" > PRIx64 "\n", ib_notice_get_type(p_ntci), > cl_ntoh16(p_ntci->g_or_v.generic. > trap_num), > + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. > + trap_num)), > cl_ntoh32(ib_notice_get_prod_type > (p_ntci)), > ib_get_producer_type_str > @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > ); > else > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " > + "Received Generic Notice type:0x%02X " > + "num:%u (%s) Producer:%u (%s) " > "from LID:%u TID:0x%016" PRIx64 > "\n", ib_notice_get_type(p_ntci), > cl_ntoh16(p_ntci->g_or_v.generic. > trap_num), > + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. > + trap_num)), > cl_ntoh32(ib_notice_get_prod_type > (p_ntci)), > ib_get_producer_type_str > -- > 1.5.4.5 > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From vlad at mellanox.co.il Mon Jul 28 07:37:20 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Mon, 28 Jul 2008 17:37:20 +0300 Subject: [ofa-general] Code drop for OFED 1.4 In-Reply-To: <200807281425.32641.olaf.kirch@oracle.com> References: <200807281425.32641.olaf.kirch@oracle.com> Message-ID: <488DD9A0.6020903@mellanox.co.il> Olaf Kirch wrote: > Hi, > > I have one more code drop for RDS in OFED 1.4. It fixes the backport > problems with RDS, which prevented it from working on anything but > mainline. > > As usual, please pull from > http://www.openfabrics.org/git/?p=~okir/ofed_1_4/linux-2.6.git code-drop-20080728 > Done, Regards, Vladimir From tziporet at mellanox.co.il Mon Jul 28 08:14:50 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 28 Jul 2008 18:14:50 +0300 Subject: [ofa-general] OFED meeting agenda for today (July 28, 2008) Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD2BA325@mtlexch01.mtl.com> This is the agenda for OFED meeting today (July 28, 2008) 1. OFED 1.4 status: Need update on all open items: - MVAPICH 1.1 - Pavel Shamis (Pasha) - MVAPICH2 1.2 - Jonathan L. Perkins - Open MPI 1.3 - Jeff Squyres - NFSoRDMA - backport to RHEL5 and SLES 10 - Jeff Backer 2. Testing description: I wish to collect data on testing that each company performs: - HW: HCA/RNIC - Switch used - CPU arch - OSes - ULPs - Apps By having this we will be able to know our testing coverage (see Mellanox report below) 3. A request to add CentOS operating systems to officially supported OSes by OFED 4. Open discussion Example for Mellanox test description: Mellanox testing coverage: HCAs: InfiniHost, InfiniHost III Ex and Lx, ConnectX CPU arch: x86, x86_64, PPC64, ia64 - compilation & install only OSes: - RedHat EL4 up4: 2.6.9-42.ELsmp - RedHat EL4 up5: 2.6.9-55.ELsmp - RedHat EL4 up6: 2.6.9-67.ELsmp - RedHat EL5: 2.6.18-8.el5 - RedHat EL5 up1: 2.6.18-53.el5 - RedHat EL5 up2: 2.6.18-92.el5 - Fedora C9: 2.6.25-14.fc9 - partial - SLES10: 2.6.16.21-0.8-smp - SLES10 SP1: 2.6.16.46-0.12-smp - SLES10 SP1 up1: 2.6.16.53-0.16-smp - SLES10 SP2: 2.6.16.60-0.21-smp - OpenSuSE 10.3: 2.6.22.5-31 - partial - kernel.org: 2.6.26 and 2.6.27 (when it will be available) ULPs: Verbs, MVAPICH, Open MPI, IPoIB, SDP, RDS, SRP initiator and target Apps: OpenSM, ibutils, performance tests, mstflint Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon at opengridcomputing.com Mon Jul 28 08:29:20 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Mon, 28 Jul 2008 10:29:20 -0500 Subject: [ofa-general] [RFC PATCH] rds: enable rdma on iWARP Message-ID: <20080728152919.GA16043@opengridcomputing.com> I am able to get rds-rdma over iWARP to mostly work. It still has a timing bug that is causing the send wr ring to get corrupted, but it will run successfully for a few seconds for rds-stress with rdma enabled. This bulk of this patch is removing the pre-existing posting of the invalidate logic and adding it prior to the fastreg send posting. The previous logic assumed that posting an invalidate to a dummy qp would successfully invalidate the entry. Unfortunately, the invalidate must be posted on the same qp as the fastreg and the pre-existing logic does not have a way to get the qp the fastreg is posted on. The rest of the patch is cleaning up miscellaneous errors. This patch is based off the future-20080715 branch. Signed-Off-By: Jon Mason diff --git a/net/rds/ib.h b/net/rds/ib.h index 9e17075..6c2656f 100644 --- a/net/rds/ib.h +++ b/net/rds/ib.h @@ -74,6 +74,7 @@ struct rds_ib_mapping { struct list_head m_list; struct rds_ib_mr * m_mr; uint32_t m_rkey; + uint32_t m_prev_rkey; struct rds_ib_scatterlist m_sg; }; diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index 9969504..70adee6 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -671,6 +671,7 @@ out: int rds_ib_conn_connect(struct rds_connection *conn) { struct rds_ib_connection *ic = conn->c_transport_data; + struct rds_ib_device *rds_ibdev; struct sockaddr_in src, dest; int ret; @@ -698,8 +699,11 @@ int rds_ib_conn_connect(struct rds_connection *conn) goto out; } + rds_ibdev = ib_get_client_data(ic->i_cm_id->device, &rds_ib_client); + /* Now check the device type and set i_iwarp */ ic->i_iwarp = (ic->i_cm_id->device->node_type == RDMA_NODE_RNIC); + ic->i_fastreg = rds_ibdev->use_fastreg; dest.sin_family = AF_INET; dest.sin_addr.s_addr = (__force u32)conn->c_faddr; diff --git a/net/rds/ib_rdma.c b/net/rds/ib_rdma.c index ee473ca..47b4e1d 100644 --- a/net/rds/ib_rdma.c +++ b/net/rds/ib_rdma.c @@ -95,13 +95,6 @@ struct rds_ib_mr_pool { unsigned long max_free_pinned; struct ib_fmr_attr fmr_attr; - /* Dummy QP used to handle invalidate for fastreg */ - struct { - struct ib_qp *qp; - struct rds_ib_inv_wr *send_wrs; - struct rds_ib_work_ring send_ring, recv_ring; - } fastreg; - struct rds_ib_mr_pool_ops *op; }; @@ -139,7 +132,6 @@ static unsigned int rds_ib_unmap_fastreg_list(struct rds_ib_mr_pool *pool, static void rds_ib_destroy_fastreg(struct rds_ib_mr_pool *pool, struct rds_ib_mr *ibmr); static int rds_ib_local_invalidate(struct rds_ib_mr_pool *pool, struct rds_ib_mapping *mapping); -static void rds_ib_inval_cq_handler(struct ib_cq *cq, void *context); static struct rds_ib_mr_pool_ops rds_ib_fmr_pool_ops = { .init = rds_ib_init_fmr, @@ -393,23 +385,8 @@ struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *rds_ibdev) pool_size, &rds_ib_fastreg_pool_ops); - if (!IS_ERR(pool)) { - /* Fill in the blanks: - * create a dummy QP to which we can post LOCAL_INV - * requests when invalidating MRs - */ - rds_ib_ring_init(&pool->fastreg.send_ring, 64); - rds_ib_ring_init(&pool->fastreg.recv_ring, 64); - pool->fastreg.qp = rds_ib_create_qp(rds_ibdev, - &pool->fastreg.send_ring, - rds_ib_inval_cq_handler, - &pool->fastreg.recv_ring, - NULL, - pool); - - if (IS_ERR(pool->fastreg.qp)) - BUG(); /* FIXME handle gracefully */ - /* FIXME allocate pool->fasteg.send_wrs */ + if (IS_ERR(pool)) { + printk("__rds_ib_create_mr_pool error\n"); } } @@ -430,10 +407,6 @@ void rds_ib_destroy_mr_pool(struct rds_ib_mr_pool *pool) rds_ib_flush_mr_pool(pool, 1); BUG_ON(atomic_read(&pool->item_count)); BUG_ON(atomic_read(&pool->free_pinned)); - - if (pool->fastreg.qp) - ib_destroy_qp(pool->fastreg.qp); - kfree(pool); } @@ -697,7 +670,7 @@ void *rds_ib_get_mr(struct scatterlist *sg, unsigned long nents, ret = pool->op->map(pool, ibmr, sg, nents); if (ret == 0) - *key_ret = ibmr->u.fmr->rkey; + *key_ret = rds_ibdev->dev->node_type == RDMA_NODE_RNIC ? ibmr->fr_mr->rkey : ibmr->u.fmr->rkey; else printk(KERN_WARNING "RDS/IB: failed to map mr (errno=%d)\n", ret); @@ -908,10 +881,13 @@ static int rds_ib_map_fastreg(struct rds_ib_mr_pool *pool, rds_ib_set_scatterlist(&mapping->m_sg, sg, sg_len); + ibmr->fr_page_shift = rds_ibdev->fmr_page_shift; /* XXX really? */ + dma_pages = rds_ib_map_scatterlist(rds_ibdev, &mapping->m_sg, ibmr->fr_page_shift); if (IS_ERR(dma_pages)) { + printk("rds_ib_map_scatterlist failed \n"); ret = PTR_ERR(dma_pages); dma_pages = NULL; goto out; @@ -923,11 +899,11 @@ static int rds_ib_map_fastreg(struct rds_ib_mr_pool *pool, } ibmr->fr_page_list_len = mapping->m_sg.dma_len; - ibmr->fr_page_shift = rds_ibdev->fmr_page_shift; /* XXX really? */ for (i = 0; i < mapping->m_sg.dma_npages; ++i) ibmr->fr_page_list->page_list[i] = dma_pages[i]; + mapping->m_prev_rkey = ibmr->fr_mr->rkey; ib_update_fast_reg_key(ibmr->fr_mr, ibmr->remap_count++); mapping->m_rkey = ibmr->fr_mr->rkey; @@ -969,7 +945,7 @@ static void rds_ib_free_fastreg(struct rds_ib_mr_pool *pool, rds_ib_local_inv_complete(pool, &ibmr->mapping, IB_WC_SUCCESS); spin_lock_irqsave(&pool->list_lock, flags); - list_add(&mapping->m_list, &pool->clean_list); + list_add(&ibmr->mapping.m_list, &pool->clean_list); spin_unlock_irqrestore(&pool->list_lock, flags); return; @@ -1053,7 +1029,8 @@ struct rds_ib_mapping *rds_ib_rdma_get_mapping(struct rds_mr *mr) /* Okay, we should register the mapping now. * Set map_seq so the flush worker knows whether a * mapping is newer */ - ibmr->map_seq = atomic_read(&ibmr->pool->flush_seq); + if (ibmr->pool) + ibmr->map_seq = atomic_read(&ibmr->pool->flush_seq); return mapping; case RDS_IB_MAP_MAPPING: @@ -1077,33 +1054,19 @@ struct rds_ib_mapping *rds_ib_rdma_get_mapping(struct rds_mr *mr) */ int rds_ib_local_invalidate(struct rds_ib_mr_pool *pool, struct rds_ib_mapping *mapping) { - struct rds_ib_inv_wr *inval; - struct ib_send_wr *failed_wr; unsigned long flags; - u32 pos; - int ret; - - if (!rds_ib_ring_alloc(&pool->fastreg.send_ring, 1, &pos)) - return 0; - inval = &pool->fastreg.send_wrs[pos]; - - memset(inval, 0, sizeof(*inval)); - inval->i_wr.wr_id = pos; - inval->i_wr.opcode = IB_WR_LOCAL_INV; - inval->i_wr.ex.invalidate_rkey = mapping->m_rkey; - inval->i_mapping = mapping; + /*FIXME - potential problem. We are invalidating the fastreg prior to mapping + * it, but it should be done on the qp being used. Unfortunately, we cannot + * get to there from here. So, lie to the state machine for now, as the + * mapping will be invalidated eventually. + */ spin_lock_irqsave(&mapping->m_lock, flags); - ret = ib_post_send(pool->fastreg.qp, &inval->i_wr, &failed_wr); - if (ret == 0) { - mapping->m_state = RDS_IB_MAP_UNMAPPING; - } else { - rds_ib_ring_unalloc(&pool->fastreg.send_ring, 1); - } + mapping->m_state = RDS_IB_MAP_UNMAPPING; spin_unlock_irqrestore(&mapping->m_lock, flags); - return ret == 0; + return 1; } void rds_ib_local_inv_complete(struct rds_ib_mr_pool *pool, @@ -1126,23 +1089,6 @@ void rds_ib_local_inv_complete(struct rds_ib_mr_pool *pool, } } -static void rds_ib_inval_cq_handler(struct ib_cq *cq, void *context) -{ - struct rds_ib_mr_pool *pool = context; - struct ib_wc wc; - - ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); - while (ib_poll_cq(cq, 1, &wc) > 0) { - struct rds_ib_inv_wr *wr; - - wr = &pool->fastreg.send_wrs[wc.wr_id]; - rds_ib_local_inv_complete(pool, wr->i_mapping, wc.status); - } - - if (waitqueue_active(&pool->flush_waitq)) - wake_up(&pool->flush_waitq); -} - void rds_ib_rdma_build_fastreg(struct ib_send_wr *wr, struct rds_ib_mapping *mapping) { struct rds_ib_mr *ibmr = mapping->m_mr; @@ -1156,6 +1102,7 @@ void rds_ib_rdma_build_fastreg(struct ib_send_wr *wr, struct rds_ib_mapping *map wr->wr.fast_reg.access_flags = IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_READ | IB_ACCESS_REMOTE_WRITE; + wr->wr.fast_reg.iova_start = 0; } void rds_ib_fast_reg_complete(struct rds_ib_mapping *mapping, int status) @@ -1206,9 +1153,6 @@ static unsigned int rds_ib_unmap_fastreg_list(struct rds_ib_mr_pool *pool, while (!list_empty(unmap_list)) { unsigned long flags; - wait_event(pool->flush_waitq, - rds_ib_ring_empty(&pool->fastreg.send_ring)); - spin_lock_irqsave(&pool->list_lock, flags); list_for_each_entry_safe(mapping, next, unmap_list, m_list) { switch (mapping->m_state) { diff --git a/net/rds/ib_recv.c b/net/rds/ib_recv.c index 9f72556..c2fb039 100644 --- a/net/rds/ib_recv.c +++ b/net/rds/ib_recv.c @@ -102,7 +102,6 @@ void rds_ib_recv_init_ring(struct rds_ib_connection *ic) sge = rds_ib_header_sge(ic, recv->r_sge); sge->addr = ic->i_recv_hdrs_dma + (i * sizeof(struct rds_header)); sge->length = sizeof(struct rds_header); - sge->lkey = ic->i_mr->lkey; } } diff --git a/net/rds/ib_send.c b/net/rds/ib_send.c index 4878d3b..870c697 100644 --- a/net/rds/ib_send.c +++ b/net/rds/ib_send.c @@ -137,6 +137,7 @@ void rds_ib_send_init_ring(struct rds_ib_connection *ic) send->s_op = NULL; send->s_mapping = NULL; + send->s_wr.next = NULL; send->s_wr.wr_id = i; send->s_wr.sg_list = send->s_sge; send->s_wr.num_sge = 1; @@ -221,6 +222,7 @@ void rds_ib_send_cq_comp_handler(struct ib_cq *cq, void *context) if (send->s_rm) rds_ib_send_unmap_rm(ic, send, wc.status); break; + case IB_WR_LOCAL_INV: case IB_WR_RDMA_WRITE: case IB_WR_RDMA_READ: /* Nothing to be done - the SG list will be unmapped @@ -233,8 +235,8 @@ void rds_ib_send_cq_comp_handler(struct ib_cq *cq, void *context) default: if (printk_ratelimit()) printk(KERN_NOTICE - "RDS/IB: %s: unexpected opcode 0x%x in WR!\n", - __FUNCTION__, send->s_wr.opcode); + "RDS/IB: %s: unexpected opcode 0x%x in WR! %d\n", + __FUNCTION__, send->s_wr.opcode, wc.opcode); break; } @@ -253,7 +255,6 @@ void rds_ib_send_cq_comp_handler(struct ib_cq *cq, void *context) if ((rm = rds_send_get_message(conn, send->s_op)) != NULL) rds_ib_send_rdma_complete(rm, wc.status); } - oldest = (oldest + 1) % ic->i_send_ring.w_nr; } @@ -866,7 +867,7 @@ int rds_ib_xmit_fastreg(struct rds_connection *conn, struct rds_mr *mr) { struct rds_ib_mapping *mapping; struct rds_ib_connection *ic = conn->c_transport_data; - struct rds_ib_send_work *send = NULL; + struct rds_ib_send_work *send1 = NULL, *send2 = NULL; struct ib_send_wr *failed_wr; u32 pos; u32 work_alloc = 0; @@ -885,31 +886,50 @@ int rds_ib_xmit_fastreg(struct rds_connection *conn, struct rds_mr *mr) * inside the fast_reg_mr WR. The key used is a rolling 8bit * counter, which should guarantee uniqueness. */ - work_alloc = rds_ib_ring_alloc(&ic->i_send_ring, 1, &pos); - if (work_alloc == 0) { + work_alloc = rds_ib_ring_alloc(&ic->i_send_ring, 2, &pos); + if (work_alloc < 2) { rds_ib_ring_unalloc(&ic->i_send_ring, work_alloc); rds_ib_stats_inc(s_ib_tx_ring_full); ret = -ENOMEM; goto out; } + send1 = &ic->i_sends[pos]; + send2 = &ic->i_sends[pos+1]; - send = &ic->i_sends[pos]; + send1->s_wr.opcode = IB_WR_LOCAL_INV; + send1->s_wr.ex.invalidate_rkey = mapping->m_prev_rkey; + send1->s_wr.send_flags = IB_SEND_SIGNALED; + send1->s_queued = jiffies; - memset(send, 0, sizeof(*send)); - rds_ib_rdma_build_fastreg(&send->s_wr, mapping); - send->s_mapping = mapping; - send->s_queued = jiffies; + rds_ib_rdma_build_fastreg(&send2->s_wr, mapping); + send2->s_wr.send_flags = IB_SEND_SIGNALED; + send2->s_mapping = mapping; + send2->s_queued = jiffies; + + + failed_wr = &send1->s_wr; + ret = ib_post_send(ic->i_cm_id->qp, &send1->s_wr, &failed_wr); + + rdsdebug("ic %p send %p (wr %p) ret %d wr %p\n", ic, + send1, &send1->s_wr, ret, failed_wr); + BUG_ON(failed_wr != &send1->s_wr); + if (ret) { + printk(KERN_WARNING "RDS/IB: fastreg ib_post_send to %u.%u.%u.%u " + "returned %d\n", NIPQUAD(conn->c_faddr), ret); + rds_ib_ring_unalloc(&ic->i_send_ring, 2); + return ret; + } - failed_wr = &send->s_wr; - ret = ib_post_send(ic->i_cm_id->qp, &send->s_wr, &failed_wr); + failed_wr = &send2->s_wr; + ret = ib_post_send(ic->i_cm_id->qp, &send2->s_wr, &failed_wr); rdsdebug("ic %p send %p (wr %p) ret %d wr %p\n", ic, - send, &send->s_wr, ret, failed_wr); - BUG_ON(failed_wr != &send->s_wr); + send2, &send2->s_wr, ret, failed_wr); + BUG_ON(failed_wr != &send2->s_wr); if (ret) { printk(KERN_WARNING "RDS/IB: fastreg ib_post_send to %u.%u.%u.%u " "returned %d\n", NIPQUAD(conn->c_faddr), ret); - rds_ib_ring_unalloc(&ic->i_send_ring, 1); + rds_ib_ring_unalloc(&ic->i_send_ring, 2); return ret; } From AHKumar at odu.edu Mon Jul 28 08:38:52 2008 From: AHKumar at odu.edu (Kumar, Amit H.) Date: Mon, 28 Jul 2008 11:38:52 -0400 Subject: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created Message-ID: Hello ... Installing OFED 1.3.1 for the first time. I get the following message on building RPMS and then installation stops. It creates Kernel-ib, kernel-ib-devel and ib-bonding rpms and then fails at the following. Running rpmbuild --rebuild --define '_topdir /var/tmp/OFED_topdir' --define 'KVERSION 2.6.18-53.1.21.el5' --define '_release 2.6.18_53.1.21.el5' --define '_prefix /usr' /share/apps/OFED-1.3.1/SRPMS/ib-bonding-0.9.0-25.src.rpm ib-bonding-debuginfo was not created I then customised the installation to exclude ib-bonding-debuginfo, installation continued and failed to build any debuginfo RPMS. Finally I could complete the installation without any debuginfo libraries. Any idea what could I be doing wrong ??? Thank you, Amit -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at mellanox.co.il Mon Jul 28 08:48:59 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Mon, 28 Jul 2008 18:48:59 +0300 Subject: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created In-Reply-To: References: Message-ID: <488DEA6B.1060409@mellanox.co.il> Kumar, Amit H. wrote: > Hello ... > > Installing OFED 1.3.1 for the first time. I get the following message on > building RPMS and then installation stops. > > It creates Kernel-ib, kernel-ib-devel and ib-bonding rpms and then fails > at the following. > > Running rpmbuild --rebuild --define '_topdir /var/tmp/OFED_topdir' > --define 'KVERSION 2.6.18-53.1.21.el5' --define '_release > 2.6.18_53.1.21.el5' --define '_prefix /usr' > /share/apps/OFED-1.3.1/SRPMS/ib-bonding-0.9.0-25.src.rpm > ib-bonding-debuginfo was not created > > I then customised the installation to exclude ib-bonding-debuginfo, > installation continued and failed to build any debuginfo RPMS. Finally I > could complete the installation without any debuginfo libraries. > > Any idea what could I be doing wrong ??? > > Thank you, > Amit > Can you please send the output of 'rpm --showrc'? Regards, Vladimir From mashirle at us.ibm.com Mon Jul 28 09:08:35 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 09:08:35 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217034477.12935.15.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217261315.9030.4.camel@IBM-29AB850785D.beaverton.ibm.com> On Sat, 2008-07-26 at 14:25 -0700, Roland Dreier wrote: > > Then we have a common approach for both nonSRQ and SRQ. We can remove > > async event handler. I have tested above patches. But I would like to > > hear your thoughts before sumitting them. > > Seems like it would end up ignoring ordering rules for handling > completions etc, and open windows for either leaking receive buffers or > use-after-free. I don't know what the motivation is: what problems do > we solve by doing this? The motivation was to release IPoIB QP resource on time especially for IPoIB nonSRQ with limited connections. I will post the patches for review to make sure there is no window for leaking receive buffers or use-after-free. Thanks Shirley From olaf.kirch at oracle.com Mon Jul 28 09:13:22 2008 From: olaf.kirch at oracle.com (Olaf Kirch) Date: Mon, 28 Jul 2008 18:13:22 +0200 Subject: [ofa-general] Re: [RFC PATCH] rds: enable rdma on iWARP In-Reply-To: <20080728152919.GA16043@opengridcomputing.com> References: <20080728152919.GA16043@opengridcomputing.com> Message-ID: <200807281813.23845.olaf.kirch@oracle.com> On Monday 28 July 2008 17:29:20 Jon Mason wrote: > This bulk of this patch is removing the pre-existing posting of the invalidate > logic and adding it prior to the fastreg send posting. The previous logic > assumed that posting an invalidate to a dummy qp would successfully invalidate > the entry. Unfortunately, the invalidate must be posted on the same qp as the > fastreg and the pre-existing logic does not have a way to get the qp the fastreg > is posted on. Then I don't see how this is going to work, ever. When the Oracle IPC creates an MR, we do not know yet with which peer it wants to use it. And in fact it may want to use the same MR with several peers... I'm not sure about that detail but I think that's the case. First off, there's a semantic snag here which I wanted to avoid by *not* pairing the inval with the remap. Essentially, if the application calls FREE_MR with the invalidate flag set, it actually expects *all* previously freed MRs to be invalidated. In the FMR world, this amounts to unmapping all FMRs on the dirty list, and batch destroying them - this means we clean up the host side data structures, then issue a SYNC_TPT, and we're done. Not fast, but if you get good batching it doesn't slow you down too much. Now when you pair remaps and invalidates, you will get MRs that are in the process of being remapped, but the LOCAL_INV hasnt completed yet - so you need to add a lot of tracking for these. I tried, and it became very ugly very fast. That's why I used separate code paths for remap and invalidate - and as far as I understand there's no problem with that. The r_key's remap counter gets incremented every time you map something, so you essentially get a different r_key each time. Am I correct that with this approach, you can have a RKEY(b, v) made up of a base stag b and a version counter v. You can map MAP RKEY(b, 0) -> some memory MAP RKEY(b, 1) -> some other memory MAP RKEY(b, 2) -> yet some other memory INVAL RKEY(b, 0) INVAL RKEY(b, 1) and so on? Or does the HCA driver keep pointers to the caller's data structures around somewhere so that repeated MAP requests without intervening INVAL would lead to corruption? If that is the case, I would leave the approach of a separate map and inval in place, because free+invalidate becomes rather simple with this: you just post all the inval requests and wait for them to complete. My original approach was rather simplistic, in that I wanted to post the INVAL request to just a single dummy QP. If that doesn't work (which I think is a deficiency of the interface) then we need to record the original rds_conn somewhere with the mapping, so that we know which QP to post it to. However, I still have doubts all of this will work very well. If you have to pipeline R_Key invalidations to a variety of QPs, you may face QPs that are heavily contended - actually so much that you may not even be able to get a single inval request onto the queue because the application keeps hogging the pipe with SENDs or other transactions. IOW a single SEND intensive application can starve another app calling FREE+invalidate almost indefinitely. Second, what do you do if a QP errors out? Does that render all R_Keys issued previously on that QP invalid? That sounds like a real knockout problem to me. Imagine 1000 processing doing RDMA, all of them busily obtaining r_keys for some mapping, and asking the remote to rdma to/from that memory. Now thanks to an application bug, *one* transfer refers to an r_key that's bogus. Now your connection goes down with remote access error. Big deal, RDS will just nix all outstanding RDMA transfers, reconnect and create a new QP. Now, if it is actually as you say and memory registrations are bound to the QP they were created on, this means all previously created mappings have been invalidated at once. What happens next? Some applications will obtain a fresh mapping and retry their RDMA. Others will have been lucky, and didn't have a RDMA in flight that was dropped on the floor - they will initiate a RDMA with a r_key that is suddenly no longer valid! Guess what happens - the connection goes down again! This looks a lot like network chernobyl to me. Or, if you will, a design flaw. A mapping obtained on a given QP should be usable with other QPs bound to the same device, and you should be able to invalidate it on any QP bound to the same device. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From yossi.openib at gmail.com Mon Jul 28 09:30:15 2008 From: yossi.openib at gmail.com (Yossi Etigin) Date: Mon, 28 Jul 2008 19:30:15 +0300 Subject: [ofa-general] [RFC] ipoib: multicast sender will leave the group when neighbour is cleaned In-Reply-To: <488776CA.6010001@gmail.com> References: <48849F59.7060502@Voltaire.COM> <488776CA.6010001@gmail.com> Message-ID: <488DF417.1010407@gmail.com> Roland, Can you please comment on this? Thanks, Yossi Yossi Etigin wrote: >> >> Would it work to have a more naive approach, and just keep track of the >> last time each send-only membership was used, and set up a timer that >> runs, say every minute, and garbage collect send-only memberships that >> are more than 2 minutes old, or something like that? >> > > How about this one: > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h > b/drivers/infiniband/ulp/ipoib/ipoib.h > index ca126fc..7a5998f 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib.h > +++ b/drivers/infiniband/ulp/ipoib/ipoib.h > @@ -88,6 +88,7 @@ enum { > IPOIB_FLAG_ADMIN_CM = 9, > IPOIB_FLAG_UMCAST = 10, > IPOIB_FLAG_CSUM = 11, > + IPOIB_MCAST_RUN_GC = 12, > > IPOIB_MAX_BACKOFF_SECONDS = 16, > > @@ -127,6 +128,7 @@ struct ipoib_mcast { > struct list_head list; > > unsigned long created; > + unsigned long used; > unsigned long backoff; > > unsigned long flags; > @@ -275,7 +277,8 @@ struct ipoib_dev_priv { > struct rb_root multicast_tree; > > struct delayed_work pkey_poll_task; > - struct delayed_work mcast_task; > + struct delayed_work mcast_join_task; > + struct delayed_work mcast_leave_task; > struct work_struct flush_task; > struct work_struct restart_task; > struct delayed_work ah_reap_task; > @@ -440,6 +443,7 @@ int ipoib_dev_init(struct net_device *dev, struct > ib_device *ca, int port); > void ipoib_dev_cleanup(struct net_device *dev); > > void ipoib_mcast_join_task(struct work_struct *work); > +void ipoib_mcast_leave_task(struct work_struct *work); > void ipoib_mcast_send(struct net_device *dev, void *mgid, struct sk_buff > *skb); > > void ipoib_mcast_restart_task(struct work_struct *work); > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c > b/drivers/infiniband/ulp/ipoib/ipoib_main.c > index 2442090..b7f4b2b 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c > @@ -990,7 +990,8 @@ static void ipoib_setup(struct net_device *dev) > > INIT_DELAYED_WORK(&priv->pkey_poll_task, ipoib_pkey_poll); > INIT_WORK(&priv->pkey_event_task, ipoib_pkey_event); > - INIT_DELAYED_WORK(&priv->mcast_task, ipoib_mcast_join_task); > + INIT_DELAYED_WORK(&priv->mcast_join_task, ipoib_mcast_join_task); > + INIT_DELAYED_WORK(&priv->mcast_leave_task, ipoib_mcast_leave_task); > INIT_WORK(&priv->flush_task, ipoib_ib_dev_flush); > INIT_WORK(&priv->restart_task, ipoib_mcast_restart_task); > INIT_DELAYED_WORK(&priv->ah_reap_task, ipoib_reap_ah); > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c > b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c > index 3f663fb..1957e93 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c > @@ -120,6 +120,7 @@ static struct ipoib_mcast *ipoib_mcast_alloc(struct > net_device *dev, > > mcast->dev = dev; > mcast->created = jiffies; > + mcast->used = jiffies; > mcast->backoff = 1; > > INIT_LIST_HEAD(&mcast->list); > @@ -389,7 +390,7 @@ static int ipoib_mcast_join_complete(int status, > mutex_lock(&mcast_mutex); > if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) > queue_delayed_work(ipoib_workqueue, > - &priv->mcast_task, 0); > + &priv->mcast_join_task, 0); > mutex_unlock(&mcast_mutex); > > if (mcast == priv->broadcast) > @@ -422,7 +423,7 @@ static int ipoib_mcast_join_complete(int status, > mutex_lock(&mcast_mutex); > spin_lock_irq(&priv->lock); > if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) > - queue_delayed_work(ipoib_workqueue, &priv->mcast_task, > + queue_delayed_work(ipoib_workqueue, &priv->mcast_join_task, > mcast->backoff * HZ); > spin_unlock_irq(&priv->lock); > mutex_unlock(&mcast_mutex); > @@ -492,7 +493,7 @@ static void ipoib_mcast_join(struct net_device *dev, > struct ipoib_mcast *mcast, > mutex_lock(&mcast_mutex); > if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) > queue_delayed_work(ipoib_workqueue, > - &priv->mcast_task, > + &priv->mcast_join_task, > mcast->backoff * HZ); > mutex_unlock(&mcast_mutex); > } > @@ -501,7 +502,7 @@ static void ipoib_mcast_join(struct net_device *dev, > struct ipoib_mcast *mcast, > void ipoib_mcast_join_task(struct work_struct *work) > { > struct ipoib_dev_priv *priv = > - container_of(work, struct ipoib_dev_priv, mcast_task.work); > + container_of(work, struct ipoib_dev_priv, mcast_join_task.work); > struct net_device *dev = priv->dev; > > if (!test_bit(IPOIB_MCAST_RUN, &priv->flags)) > @@ -530,7 +531,7 @@ void ipoib_mcast_join_task(struct work_struct *work) > mutex_lock(&mcast_mutex); > if (test_bit(IPOIB_MCAST_RUN, &priv->flags)) > queue_delayed_work(ipoib_workqueue, > - &priv->mcast_task, HZ); > + &priv->mcast_join_task, HZ); > mutex_unlock(&mcast_mutex); > return; > } > @@ -591,7 +592,9 @@ int ipoib_mcast_start_thread(struct net_device *dev) > > mutex_lock(&mcast_mutex); > if (!test_and_set_bit(IPOIB_MCAST_RUN, &priv->flags)) > - queue_delayed_work(ipoib_workqueue, &priv->mcast_task, 0); > + queue_delayed_work(ipoib_workqueue, &priv->mcast_join_task, 0); > + if (!test_and_set_bit(IPOIB_MCAST_RUN_GC, &priv->flags)) > + queue_delayed_work(ipoib_workqueue, &priv->mcast_leave_task, 0); > mutex_unlock(&mcast_mutex); > > spin_lock_irq(&priv->lock); > @@ -613,7 +616,9 @@ int ipoib_mcast_stop_thread(struct net_device *dev, > int flush) > > mutex_lock(&mcast_mutex); > clear_bit(IPOIB_MCAST_RUN, &priv->flags); > - cancel_delayed_work(&priv->mcast_task); > + clear_bit(IPOIB_MCAST_RUN_GC, &priv->flags); > + cancel_delayed_work(&priv->mcast_join_task); > + cancel_delayed_work(&priv->mcast_leave_task); > mutex_unlock(&mcast_mutex); > > if (flush) > @@ -720,6 +725,7 @@ out: > } > } > > + mcast->used = jiffies; > ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN); > } > > @@ -864,6 +870,33 @@ void ipoib_mcast_restart_task(struct work_struct > *work) > ipoib_mcast_start_thread(dev); > } > > +void ipoib_mcast_leave_task(struct work_struct *work) > +{ > + struct ipoib_dev_priv *priv = > + container_of(work, struct ipoib_dev_priv, mcast_leave_task.work); > + struct net_device *dev = priv->dev; > + struct ipoib_mcast *mcast, *tmcast; > + LIST_HEAD(remove_list); > + > + if (!test_bit(IPOIB_MCAST_RUN_GC, &priv->flags)) > + return; > + > + list_for_each_entry_safe(mcast, tmcast, &priv->multicast_list, list) { > + if (test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags) && > + time_before(mcast->used, jiffies - 120 * HZ)) { > + rb_erase(&mcast->rb_node, &priv->multicast_tree); > + list_move_tail(&mcast->list, &remove_list); > + } > + } > + > + list_for_each_entry_safe(mcast, tmcast, &remove_list, list) { > + ipoib_mcast_leave(dev, mcast); > + ipoib_mcast_free(mcast); > + } > + > + queue_delayed_work(ipoib_workqueue, &priv->mcast_leave_task, 60 * HZ); > +} > + > #ifdef CONFIG_INFINIBAND_IPOIB_DEBUG > > struct ipoib_mcast_iter *ipoib_mcast_iter_init(struct net_device *dev) > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From sashak at voltaire.com Mon Jul 28 09:41:06 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 28 Jul 2008 19:41:06 +0300 Subject: [ofa-general] Re: [PATCH] complib/cl_event_wheel.c: Fix some printf typos In-Reply-To: References: Message-ID: <20080728164106.GN14872@sashak.voltaire.com> On 09:54 Mon 28 Jul , Hal Rosenstock wrote: > complib/cl_event_wheel.c: Fix some printf typos > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Mon Jul 28 09:45:48 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 28 Jul 2008 19:45:48 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_lid_mgr.c: Convert lid range prints to decimal In-Reply-To: References: Message-ID: <20080728164548.GO14872@sashak.voltaire.com> On 09:54 Mon 28 Jul , Hal Rosenstock wrote: > opensm/osm_lid_mgr.c: Convert lid range prints to decimal > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From Jeffrey.C.Becker at nasa.gov Mon Jul 28 09:51:07 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Mon, 28 Jul 2008 09:51:07 -0700 Subject: [ofa-general] [Fwd: Re: NFS-RDMA backport question] Message-ID: <488DF8FB.20508@nasa.gov> Hi. Any advice on backporting issue below? Thanks. -jeff -------- Original Message -------- Subject: Re: NFS-RDMA backport question Date: Mon, 28 Jul 2008 10:11:07 -0400 (EDT) From: James Lentini To: Jeff Becker CC: Tom Tucker , Thomas Talpey References: <488A1DFD.8020301 at nasa.gov> On Fri, 25 Jul 2008, Jeff Becker wrote: > Hi. I was doing the backport to 2.6.16.60 (for sles 10 sp2), and I > hit a compile problem with launder_page in fs/nfs/file.c. I looked > it up in the git commits, and saw that Trond added it to 2.6.20 to > fix a race in nfs_release_page(). The problem for backports is that > the fix also adds code to mm/truncate.c. If my understanding is > correct, the whole point of the backports is to have modules that > will work with older kernels (e.g. from distros). However, if I > changed mm/truncate.c, the kernel would have to get recompiled, > which is a more drastic change than running a backported set of > NFS-RDMA modules. Thus, am I stuck leaving the race in kernels older > than 2.6.20? Please advise. Thanks. > > -jeff Jeff, It sounds like you have a handle on this from an NFS perspective. For this sort of judgment call, I think you should bring this up with the OFED maintainers. I expect that they have a policy for handling these sorts of situations. james From sashak at voltaire.com Mon Jul 28 09:52:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 28 Jul 2008 19:52:32 +0300 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: References: <20080709204617.GR12551@sashak.voltaire.com> <1215636792.18371.129.camel@hrosenstock-ws.xsigo.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> <20080724023252.GB11924@sashak.voltaire.com> <20080724031356.GD11924@sashak.voltaire.com> Message-ID: <20080728165232.GP14872@sashak.voltaire.com> On 09:59 Mon 28 Jul , Hal Rosenstock wrote: > > > Anyway I see this issue as pretty minor. It is fixed now > > Is the version in libibmad.map updated now ? It is symbol versioning, not a library. > > and will appear in next tarball release. > > What is your "policy" for updating the various versions on an API update ? We are updating library version (with -version-info) when API changes (additions are included). I think it is fine to do it once before tarball release and not for each single change. Sasha From mashirle at us.ibm.com Mon Jul 28 10:01:20 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 10:01:20 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217039352.12935.24.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217264480.9030.32.camel@IBM-29AB850785D.beaverton.ibm.com> > I don't understand... are you saying that pending receives don't get > completed with a flush error when a QP transitions to error state? I would say so since I didn't see the QP got destroyed. You meant susequents WQEs need to return flush error completion status throught CQ? Current ehca code level doesn't have this support. It will be fixed later. Thanks Shirley From weiny2 at llnl.gov Mon Jul 28 10:53:16 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 28 Jul 2008 10:53:16 -0700 Subject: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> <20080724101113.76537e5e.weiny2@llnl.gov> Message-ID: <20080728105316.16cd4bf9.weiny2@llnl.gov> On Mon, 28 Jul 2008 10:02:15 -0400 "Hal Rosenstock" wrote: > On Thu, Jul 24, 2008 at 1:11 PM, Ira Weiny wrote: > > >From 1f810dd6ae9a73549cacdbb26f464e6132675592 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Tue, 22 Jul 2008 16:31:36 -0700 > > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM > > > > > > Signed-off-by: Ira K. Weiny > > --- > > opensm/include/iba/ib_types.h | 2 + > > opensm/opensm/libopensm.map | 1 + > > opensm/opensm/osm_helper.c | 44 +++++++++++++++++++++++++++++++++++++++++ > > opensm/opensm/osm_inform.c | 3 +- > > opensm/opensm/osm_trap_rcv.c | 10 +++++++- > > 5 files changed, 57 insertions(+), 3 deletions(-) > > > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > > index 09ec257..fffce00 100644 > > --- a/opensm/include/iba/ib_types.h > > +++ b/opensm/include/iba/ib_types.h > > @@ -7219,6 +7219,8 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) > > #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) > > > > +char * OSM_API ib_get_trap_str(uint16_t trap_num); > > + > > /****f* IBA Base: Types/ib_notice_is_generic > > * NAME > > * ib_notice_is_generic > > diff --git a/opensm/opensm/libopensm.map b/opensm/opensm/libopensm.map > > index 759a4e8..7cd2aba 100644 > > --- a/opensm/opensm/libopensm.map > > +++ b/opensm/opensm/libopensm.map > > @@ -17,6 +17,7 @@ OPENSM_1.5 { > > Doesn't this also need updating as an API is being added ? > > -- Hal Probably, do we have anything else we want to add/change? Ira > > > ib_get_sm_method_str; > > ib_get_sm_attr_str; > > ib_get_sa_attr_str; > > + ib_get_trap_str; > > osm_dump_port_info; > > osm_dump_portinfo_record; > > osm_dump_guidinfo_record; > > diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c > > index 21ff51c..cc76d5c 100644 > > --- a/opensm/opensm/osm_helper.c > > +++ b/opensm/opensm/osm_helper.c > > @@ -2338,3 +2338,47 @@ const char *osm_get_sm_mgr_state_str(IN uint16_t state) > > __osm_sm_mgr_state_str[state] : > > __osm_sm_mgr_state_str[ARR_SIZE(__osm_sm_mgr_state_str) - 1]; > > } > > + > > +char * OSM_API > > +ib_get_trap_str(uint16_t trap_num) > > +{ > > + switch(trap_num) > > + { > > + case 64: > > + return ("GID in service"); > > + case 65: > > + return ("GID out of service"); > > + case 66: > > + return ("New mcast group created"); > > + case 67: > > + return ("Mcast group deleted"); > > + case 68: > > + return ("UnPath, Path no longer valid"); > > + case 69: > > + return ("RePath, Path recomputed"); > > + case 128: > > + return ("Link state change"); > > + case 129: > > + return ("Local Link integrity threshold reached"); > > + case 130: > > + return ("Excessive Buffer Overrun Threshold reached"); > > + case 131: > > + return ("Flow Control Update watchdog timer expired"); > > + case 144: > > + return ("CapabilityMask, NodeDescription, Link [Width|Speed] Enabled changed"); > > + case 145: > > + return ("System Image GUID changed"); > > + case 256: > > + return ("Bad M_Key"); > > + case 257: > > + return ("Bad P_Key"); > > + case 258: > > + return ("Bad Q_Key"); > > + case 259: > > + return ("Bad P_Key (switch external port)"); > > + default: > > + break; > > + } > > + return ("Unknown"); > > +} > > + > > diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c > > index e6146f7..ebc13bb 100644 > > --- a/opensm/opensm/osm_inform.c > > +++ b/opensm/opensm/osm_inform.c > > @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, > > /* an official Event information log */ > > if (ib_notice_is_generic(p_ntc)) { > > OSM_LOG(p_log, OSM_LOG_INFO, > > - "Reporting Generic Notice type:%u num:%u" > > + "Reporting Generic Notice type:%u num:%u (%s)" > > " from LID:%u GID:0x%016" PRIx64 > > ",0x%016" PRIx64 "\n", > > ib_notice_get_type(p_ntc), > > cl_ntoh16(p_ntc->g_or_v.generic.trap_num), > > + ib_get_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), > > cl_ntoh16(p_ntc->issuer_lid), > > cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), > > cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) > > diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c > > index ae7ab05..03cbd26 100644 > > --- a/opensm/opensm/osm_trap_rcv.c > > +++ b/opensm/opensm/osm_trap_rcv.c > > @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > > || (p_ntci->g_or_v.generic.trap_num == > > CL_HTON16(131))) > > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " > > + "Received Generic Notice type:0x%02X " > > + "num:%u (%s) Producer:%u (%s) " > > "from LID:%u Port %d TID:0x%016" > > PRIx64 "\n", ib_notice_get_type(p_ntci), > > cl_ntoh16(p_ntci->g_or_v.generic. > > trap_num), > > + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. > > + trap_num)), > > cl_ntoh32(ib_notice_get_prod_type > > (p_ntci)), > > ib_get_producer_type_str > > @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > > ); > > else > > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " > > + "Received Generic Notice type:0x%02X " > > + "num:%u (%s) Producer:%u (%s) " > > "from LID:%u TID:0x%016" PRIx64 > > "\n", ib_notice_get_type(p_ntci), > > cl_ntoh16(p_ntci->g_or_v.generic. > > trap_num), > > + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. > > + trap_num)), > > cl_ntoh32(ib_notice_get_prod_type > > (p_ntci)), > > ib_get_producer_type_str > > -- > > 1.5.4.5 > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http:// lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http:// openib.org/mailman/listinfo/openib-general > > > From swise at opengridcomputing.com Mon Jul 28 10:56:00 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 28 Jul 2008 12:56:00 -0500 Subject: [ofa-general] Re: [RFC PATCH] rds: enable rdma on iWARP In-Reply-To: <200807281813.23845.olaf.kirch@oracle.com> References: <20080728152919.GA16043@opengridcomputing.com> <200807281813.23845.olaf.kirch@oracle.com> Message-ID: <488E0830.1050802@opengridcomputing.com> Olaf Kirch wrote: > On Monday 28 July 2008 17:29:20 Jon Mason wrote: > >> This bulk of this patch is removing the pre-existing posting of the invalidate >> logic and adding it prior to the fastreg send posting. The previous logic >> assumed that posting an invalidate to a dummy qp would successfully invalidate >> the entry. Unfortunately, the invalidate must be posted on the same qp as the >> fastreg and the pre-existing logic does not have a way to get the qp the fastreg >> is posted on. >> > > This isn't quite correct. The invalidate must be posted on a connected qp in the same pd. But it doesn't have to be the same qp as the fastreg. However if you use different qps, then you must coordinate that you're done using the mr before invalidating it. > Then I don't see how this is going to work, ever. When the Oracle IPC > creates an MR, we do not know yet with which peer it wants to use it. > And in fact it may want to use the same MR with several peers... I'm > not sure about that detail but I think that's the case. > > First off, there's a semantic snag here which I wanted to avoid by > *not* pairing the inval with the remap. Essentially, if the application > calls FREE_MR with the invalidate flag set, it actually expects *all* > previously freed MRs to be invalidated. In the FMR world, this > amounts to unmapping all FMRs on the dirty list, and batch destroying > them - this means we clean up the host side data structures, then issue > a SYNC_TPT, and we're done. Not fast, but if you get good batching it > doesn't slow you down too much. > > Now when you pair remaps and invalidates, you will get MRs that are > in the process of being remapped, but the LOCAL_INV hasnt completed > yet - so you need to add a lot of tracking for these. I tried, > and it became very ugly very fast. That's why I used separate code > paths for remap and invalidate - and as far as I understand there's > no problem with that. The r_key's remap counter gets incremented > every time you map something, so you essentially get a different > r_key each time. Am I correct that with this approach, you can have > a RKEY(b, v) made up of a base stag b and a version counter v. > You can map > MAP RKEY(b, 0) -> some memory > MAP RKEY(b, 1) -> some other memory > MAP RKEY(b, 2) -> yet some other memory > INVAL RKEY(b, 0) > INVAL RKEY(b, 1) > No you must invalidate the MR between fastreg calls. Like this: FASTREG RKEY(b, 0) -> some memory INVALIDATE RKEY(b, 0) FASTREG RKEY(b, 1) -> some other memory INVALIDATE RKEY(b, 1) FASTREG RKEY(b, 2) -> yet some other memory If you post them all on the same QP then you can use fencing to keep the pipeline full. If you want to use different qps for the invalidates, then you must manage that you invalidate only when you're done using them. > and so on? Or does the HCA driver keep pointers to the caller's > data structures around somewhere so that repeated MAP requests > without intervening INVAL would lead to corruption? > It doesn't have to do with the callers data structs. The simple fact is, you cannot fast register a mr to more than one pbl at a time. If you think about adapter resources, there is a single MR entry for the fast reg MR and one PBL entry for whatever the pbl is for that current mapping. > If that is the case, I would leave the approach of a separate map > and inval in place, because free+invalidate becomes rather simple > with this: you just post all the inval requests and wait for them > to complete. > Note you can just dereg the MR to invalidate the last mapping. IE you don't need to post an invalidate if you are going to call ib_dereg_mr() to destroy the fast reg mr. > My original approach was rather simplistic, in that I wanted to > post the INVAL request to just a single dummy QP. If that doesn't > work (which I think is a deficiency of the interface) then we > need to record the original rds_conn somewhere with the mapping, > so that we know which QP to post it to. > First off, any send WR posted to a QP not in RTS does nothing. For iWARP QPs, you only enter into RTS when the QP is connected. So a dummy QP just won't work. We can argure about the interface deficiencies if you want, but the semantics are part of the IBTA and iWARP specs, so we probably shouldn't change it much. > However, I still have doubts all of this will work very well. > If you have to pipeline R_Key invalidations to a variety of QPs, > you may face QPs that are heavily contended - actually so much > that you may not even be able to get a single inval request onto > the queue because the application keeps hogging the pipe with SENDs > or other transactions. IOW a single SEND intensive application can > starve another app calling FREE+invalidate almost indefinitely. > > I would think a single SEND intensive app could starve other apps trying to use the same QP anyway, so you must have some sort of fairness logic, eh? > Second, what do you do if a QP errors out? Does that render all > R_Keys issued previously on that QP invalid? No. the R_Key's aren't tied to the QP except that if you have pending fastreg or invalidate WRS, then it is tied to that QP until the WRs complete. > That sounds like a > real knockout problem to me. Imagine 1000 processing doing RDMA, > all of them busily obtaining r_keys for some mapping, and asking > the remote to rdma to/from that memory. Now thanks to an application > bug, *one* transfer refers to an r_key that's bogus. Now your > connection goes down with remote access error. Big deal, RDS > will just nix all outstanding RDMA transfers, reconnect and create > a new QP. > > Now, if it is actually as you say and memory registrations are > bound to the QP they were created on, this means all previously > created mappings have been invalidated at once. What happens next? > Some applications will obtain a fresh mapping and retry their > RDMA. Others will have been lucky, and didn't have a RDMA in flight > that was dropped on the floor - they will initiate a RDMA with > a r_key that is suddenly no longer valid! Guess what happens - the > connection goes down again! > > This looks a lot like network chernobyl to me. > Or, if you will, a design flaw. A mapping obtained on a > given QP should be usable with other QPs bound to the same > device, and you should be able to invalidate it on any QP > bound to the same device. > > It is. You can. > Olaf > From rdreier at cisco.com Mon Jul 28 11:33:27 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 28 Jul 2008 11:33:27 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1217264480.9030.32.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Mon, 28 Jul 2008 10:01:20 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217039352.12935.24.camel@IBM-29AB850785D.beaverton.ibm.com> <1217264480.9030.32.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > I would say so since I didn't see the QP got destroyed. You meant > susequents WQEs need to return flush error completion status throught > CQ? Current ehca code level doesn't have this support. It will be fixed > later. That seems like the real issue. If ehca does not generate flush error completions for pending work requests when a QP is transitioned to the error state, then that is a major problem that is going to break cleanup for nearly everything. Generating flush error completions on transition to error is a fundamental part of the IB verbs. So I would prefer not to work around it in IPoIB but rather fix this basic problem in ehca. - R. From rdreier at cisco.com Mon Jul 28 11:34:02 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 28 Jul 2008 11:34:02 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <1217261315.9030.4.camel@IBM-29AB850785D.beaverton.ibm.com> (Shirley Ma's message of "Mon, 28 Jul 2008 09:08:35 -0700") References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217034477.12935.15.camel@IBM-29AB850785D.beaverton.ibm.com> <1217261315.9030.4.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: > The motivation was to release IPoIB QP resource on time especially for > IPoIB nonSRQ with limited connections. I will post the patches for > review to make sure there is no window for leaking receive buffers or > use-after-free. I don't see how the changes help with this. From mashirle at us.ibm.com Mon Jul 28 11:51:37 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 11:51:37 -0700 Subject: [ofa-general] [RFC][PATCH] IPoIB: release RC QP resource during connection establishment Message-ID: <1217271097.18362.14.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, Here is a patch to release QP resources in the context of before RC QP establishement. Since the send QP hasn't set to RTU, there will be no incoming packets to the receive QP, so it's safe to release this kind of QP resource immediately. It replaces current approach: putting QP in error status and waiting for all WQEs flushing in CQ to free QP resource. This approach reduces flushing CQEs overhead and also avoid potential resource leakage: like modify_qp failure, failed to put QP in RTS status, thus no drain WRs being generated... Signed-off-by: Shirley Ma ------ drivers/infiniband/ulp/ipoib/ipoib_cm.c | 19 ++++++++++++------- 1 files changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 0f2d304..90b8668 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -480,8 +480,10 @@ static int ipoib_cm_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *even ret = ipoib_cm_send_rep(dev, cm_id, p->qp, &event->param.req_rcvd, psn); if (ret) { ipoib_warn(priv, "failed to send REP: %d\n", ret); - if (ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE)) - ipoib_warn(priv, "unable to move qp to error state\n"); + spin_lock_irqsave(&priv->lock, flags); + list_move(&p->list, &priv->cm.rx_reap_list); + spin_unlock_irqrestore(&priv->lock, flags); + queue_work(ipoib_workqueue, &priv->cm.rx_reap_task); } return 0; @@ -497,19 +499,22 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, { struct ipoib_cm_rx *p; struct ipoib_dev_priv *priv; + p = cm_id->context; + priv = netdev_priv(p->dev); switch (event->event) { case IB_CM_REQ_RECEIVED: return ipoib_cm_req_handler(cm_id, event); case IB_CM_DREQ_RECEIVED: - p = cm_id->context; ib_send_cm_drep(cm_id, NULL, 0); - /* Fall through */ - case IB_CM_REJ_RECEIVED: - p = cm_id->context; - priv = netdev_priv(p->dev); if (ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE)) ipoib_warn(priv, "unable to move qp to error state\n"); + return 0; + case IB_CM_REJ_RECEIVED: + spin_lock_irqsave(&priv->lock, flags); + list_move(&p->list, &priv->cm.rx_reap_list); + spin_unlock_irqrestore(&priv->lock, flags); + queue_work(ipoib_workqueue, &priv->cm.rx_reap_task); /* Fall through */ default: return 0; From hal.rosenstock at gmail.com Mon Jul 28 12:54:59 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 15:54:59 -0400 Subject: [ofa-general] [PATCH] opensm/osm_lid_mgr.c: Cosmetic format change Message-ID: opensm/osm_lid_mgr.c: Cosmetic format change Fix missing left brace introduced by previous patch Signed-off-by: Hal Rosenstock --- diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c index 0b6f29c..7b1aaef 100644 --- a/opensm/opensm/osm_lid_mgr.c +++ b/opensm/opensm/osm_lid_mgr.c @@ -820,7 +820,7 @@ __osm_lid_mgr_get_port_lid(IN osm_lid_mgr_t * const p_mgr, } else OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG, "0x%016" PRIx64 " existing lid range:" - "%u:%u] is not lmc aligned\n", + "[%u:%u] is not lmc aligned\n", guid, min_lid, min_lid + num_lids - 1); } From hal.rosenstock at gmail.com Mon Jul 28 12:55:12 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 15:55:12 -0400 Subject: [ewg] Re: [ofa-general] [ANNOUNCE] management tarballs release In-Reply-To: <20080728165232.GP14872@sashak.voltaire.com> References: <20080709204617.GR12551@sashak.voltaire.com> <20080709211756.GS12551@sashak.voltaire.com> <1215693745.18371.144.camel@hrosenstock-ws.xsigo.com> <20080723235538.GL10706@sashak.voltaire.com> <20080724023252.GB11924@sashak.voltaire.com> <20080724031356.GD11924@sashak.voltaire.com> <20080728165232.GP14872@sashak.voltaire.com> Message-ID: On Mon, Jul 28, 2008 at 12:52 PM, Sasha Khapyorsky wrote: > On 09:59 Mon 28 Jul , Hal Rosenstock wrote: >> >> > Anyway I see this issue as pretty minor. It is fixed now >> >> Is the version in libibmad.map updated now ? > > It is symbol versioning, not a library. >> > and will appear in next tarball release. >> >> What is your "policy" for updating the various versions on an API update ? > > We are updating library version (with -version-info) when API changes > (additions are included). I think it is fine to do it once before tarball > release and not for each single change. Yes that is fine as long as it is not forgotten. -- Hal > Sasha > From mashirle at us.ibm.com Mon Jul 28 13:05:12 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 13:05:12 -0700 Subject: [ofa-general] [RFC][PATCH] IPoIB: release QP resources in error_list when maxium connections reached Message-ID: <1217275512.18362.25.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, Here is the patch to release QP resources in error status when nonSRQ maxium connections reached. QP will be put on the error list when DREQ has received or a connection becomes stale for any reason. When the maxium nonSRQ connection has been reached, for any new active connection, if the QP on the error list has been more than 10 secs, we can safely destroy this kind of QP, and the new connection should be successful instead of being rejected. Signed-off-by: Shirley Ma ------ drivers/infiniband/ulp/ipoib/ipoib_cm.c | 40 +++++++++++++++++++++++++++++++ 1 files changed, 40 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 90b8668..d3a8540 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -341,6 +341,24 @@ static void ipoib_cm_init_rx_wr(struct net_device *dev, wr->num_sge = priv->cm.num_frags; } +static void ipoib_cm_free_rx_reap_list(struct net_device *dev); + +static void ipoib_cm_nonsrq_gc_connections(struct ipoib_dev_priv *priv) +{ + struct ipoib_cm_rx *p; + while (!list_empty(&priv->cm.rx_error_list)) { + /* List is sorted by LRU, start from tail, + * stop when we see a recently used entry */ + p = list_entry(priv->cm.rx_error_list.prev, typeof(*p), list); + /* recyle QPs which in error status more than 10 secs */ + if (time_before_eq(jiffies, p->jiffies + IPOIB_CM_RX_TIMEOUT + + 10 * HZ)); + break; + list_move(&p->list, &priv->cm.rx_reap_list); + } + return; +} + static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_id, struct ipoib_cm_rx *rx) { @@ -352,6 +370,22 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i int ret; int i; + + spin_lock_irq(&priv->lock); + if (priv->cm.nonsrq_conn_qp >= ipoib_max_conn_qp) { + if (!list_empty(&priv->cm.rx_error_list)) + ipoib_cm_nonsrq_gc_connections(priv); + if (list_empty(&priv->cm.rx_reap_list)) { + spin_unlock_irq(&priv->lock); + printk(KERN_WARNING "Max nonsrq QP exceeded. Will REJ\n"); + ib_send_cm_rej(cm_id, IB_CM_REJ_NO_QP, NULL, 0, NULL, 0); + return -EINVAL; + } else { + spin_unlock_irq(&priv->lock); + ipoib_cm_free_rx_reap_list(dev); + } + } + rx->rx_ring = kcalloc(ipoib_recvq_size, sizeof *rx->rx_ring, GFP_KERNEL); if (!rx->rx_ring) return -ENOMEM; @@ -438,6 +472,7 @@ static int ipoib_cm_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *even struct ipoib_cm_rx *p; unsigned psn; int ret; + unsigned long flags; ipoib_dbg(priv, "REQ arrived\n"); p = kzalloc(sizeof *p, GFP_KERNEL); @@ -499,6 +534,7 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, { struct ipoib_cm_rx *p; struct ipoib_dev_priv *priv; + unsigned long flags; p = cm_id->context; priv = netdev_priv(p->dev); @@ -507,6 +543,10 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, return ipoib_cm_req_handler(cm_id, event); case IB_CM_DREQ_RECEIVED: ib_send_cm_drep(cm_id, NULL, 0); + spin_lock_irq(&priv->lock); + list_move(&p->list, &priv->cm.rx_error_list); + p->state = IPOIB_CM_RX_ERROR; + spin_unlock_irq(&priv->lock); if (ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE)) ipoib_warn(priv, "unable to move qp to error state\n"); return 0; From mashirle at us.ibm.com Mon Jul 28 13:48:40 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 13:48:40 -0700 Subject: [ofa-general] [RFC][PATCH] IPoIB: release RC QP resource during connection establishment In-Reply-To: <1217271097.18362.14.camel@IBM-29AB850785D.beaverton.ibm.com> References: <1217271097.18362.14.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217278120.28057.1.camel@IBM-29AB850785D.beaverton.ibm.com> Repost with the right format. Signed-off-by: Shirley Ma ------ drivers/infiniband/ulp/ipoib/ipoib_cm.c | 19 ++++++++++++------- 1 files changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 0f2d304..90b8668 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -480,8 +480,10 @@ static int ipoib_cm_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *even ret = ipoib_cm_send_rep(dev, cm_id, p->qp, &event->param.req_rcvd, psn); if (ret) { ipoib_warn(priv, "failed to send REP: %d\n", ret); - if (ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE)) - ipoib_warn(priv, "unable to move qp to error state\n"); + spin_lock_irqsave(&priv->lock, flags); + list_move(&p->list, &priv->cm.rx_reap_list); + spin_unlock_irqrestore(&priv->lock, flags); + queue_work(ipoib_workqueue, &priv->cm.rx_reap_task); } return 0; @@ -497,19 +499,22 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, { struct ipoib_cm_rx *p; struct ipoib_dev_priv *priv; + p = cm_id->context; + priv = netdev_priv(p->dev); switch (event->event) { case IB_CM_REQ_RECEIVED: return ipoib_cm_req_handler(cm_id, event); case IB_CM_DREQ_RECEIVED: - p = cm_id->context; ib_send_cm_drep(cm_id, NULL, 0); - /* Fall through */ - case IB_CM_REJ_RECEIVED: - p = cm_id->context; - priv = netdev_priv(p->dev); if (ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE)) ipoib_warn(priv, "unable to move qp to error state\n"); + return 0; + case IB_CM_REJ_RECEIVED: + spin_lock_irqsave(&priv->lock, flags); + list_move(&p->list, &priv->cm.rx_reap_list); + spin_unlock_irqrestore(&priv->lock, flags); + queue_work(ipoib_workqueue, &priv->cm.rx_reap_task); /* Fall through */ default: return 0; From mashirle at us.ibm.com Mon Jul 28 13:50:17 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 13:50:17 -0700 Subject: [ofa-general] [RFC][PATCH] IPoIB: release QP resources in error_list when maxium connections reached In-Reply-To: <1217275512.18362.25.camel@IBM-29AB850785D.beaverton.ibm.com> References: <1217275512.18362.25.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217278217.28057.4.camel@IBM-29AB850785D.beaverton.ibm.com> Repost it with the right format. Signed-off-by: Shirley Ma -------- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 40 +++++++++++++++++++++++++++++++ 1 files changed, 40 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 90b8668..d3a8540 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -341,6 +341,24 @@ static void ipoib_cm_init_rx_wr(struct net_device *dev, wr->num_sge = priv->cm.num_frags; } +static void ipoib_cm_free_rx_reap_list(struct net_device *dev); + +static void ipoib_cm_nonsrq_gc_connections(struct ipoib_dev_priv *priv) +{ + struct ipoib_cm_rx *p; + while (!list_empty(&priv->cm.rx_error_list)) { + /* List is sorted by LRU, start from tail, + * stop when we see a recently used entry */ + p = list_entry(priv->cm.rx_error_list.prev, typeof(*p), list); + /* recyle QPs which in error status more than 10 secs */ + if (time_before_eq(jiffies, p->jiffies + IPOIB_CM_RX_TIMEOUT + + 10 * HZ)); + break; + list_move(&p->list, &priv->cm.rx_reap_list); + } + return; +} + static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_id, struct ipoib_cm_rx *rx) { @@ -352,6 +370,22 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i int ret; int i; + + spin_lock_irq(&priv->lock); + if (priv->cm.nonsrq_conn_qp >= ipoib_max_conn_qp) { + if (!list_empty(&priv->cm.rx_error_list)) + ipoib_cm_nonsrq_gc_connections(priv); + if (list_empty(&priv->cm.rx_reap_list)) { + spin_unlock_irq(&priv->lock); + printk(KERN_WARNING "Max nonsrq QP exceeded. Will REJ\n"); + ib_send_cm_rej(cm_id, IB_CM_REJ_NO_QP, NULL, 0, NULL, 0); + return -EINVAL; + } else { + spin_unlock_irq(&priv->lock); + ipoib_cm_free_rx_reap_list(dev); + } + } + rx->rx_ring = kcalloc(ipoib_recvq_size, sizeof *rx->rx_ring, GFP_KERNEL); if (!rx->rx_ring) return -ENOMEM; @@ -438,6 +472,7 @@ static int ipoib_cm_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *even struct ipoib_cm_rx *p; unsigned psn; int ret; + unsigned long flags; ipoib_dbg(priv, "REQ arrived\n"); p = kzalloc(sizeof *p, GFP_KERNEL); @@ -499,6 +534,7 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, { struct ipoib_cm_rx *p; struct ipoib_dev_priv *priv; + unsigned long flags; p = cm_id->context; priv = netdev_priv(p->dev); @@ -507,6 +543,10 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, return ipoib_cm_req_handler(cm_id, event); case IB_CM_DREQ_RECEIVED: ib_send_cm_drep(cm_id, NULL, 0); + spin_lock_irq(&priv->lock); + list_move(&p->list, &priv->cm.rx_error_list); + p->state = IPOIB_CM_RX_ERROR; + spin_unlock_irq(&priv->lock); if (ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE)) ipoib_warn(priv, "unable to move qp to error state\n"); return 0; From mashirle at us.ibm.com Mon Jul 28 13:57:07 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 13:57:07 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217034477.12935.15.camel@IBM-29AB850785D.beaverton.ibm.com> <1217261315.9030.4.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217278627.28057.12.camel@IBM-29AB850785D.beaverton.ibm.com> On Mon, 2008-07-28 at 11:34 -0700, Roland Dreier wrote: > I don't see how the changes help with this. I modified the patch based on our discussion here. Please review these two patches to see the idea is good: [RFC][PATCH] IPoIB: release RC QP resource during connection establishment This will destroy QP resources immediately since the send/recv QP connection has not been fully established. [RFC][PATCH] IPoIB: release QP resources in error_list when maxium connections This allows IPoIB nonSRQ keeping connectivity when there are QPs in error_list and maximum connections are reached. thanks Shirley From mashirle at us.ibm.com Mon Jul 28 13:58:22 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Mon, 28 Jul 2008 13:58:22 -0700 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: <1216772482.31058.77.camel@IBM-29AB850785D.beaverton.ibm.com> <1216787431.31058.96.camel@IBM-29AB850785D.beaverton.ibm.com> <1216792866.31058.135.camel@IBM-29AB850785D.beaverton.ibm.com> <1216850239.31058.145.camel@IBM-29AB850785D.beaverton.ibm.com> <1216859055.31058.170.camel@IBM-29AB850785D.beaverton.ibm.com> <1217030193.12935.2.camel@IBM-29AB850785D.beaverton.ibm.com> <1217039352.12935.24.camel@IBM-29AB850785D.beaverton.ibm.com> <1217264480.9030.32.camel@IBM-29AB850785D.beaverton.ibm.com> Message-ID: <1217278702.28057.13.camel@IBM-29AB850785D.beaverton.ibm.com> Hello Roland, > So I would prefer not to work around it in IPoIB but rather fix this > basic problem in ehca. Agree. I will not put a workaround fix in IPoIB, I have modified the patch. Thanks Shirley From chu11 at llnl.gov Mon Jul 28 13:59:19 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 28 Jul 2008 13:59:19 -0700 Subject: [ofa-general] [OpenSM][Trivial] Change --ucast_file to --ucast_dump_file Message-ID: <1217278759.16508.360.camel@cardanus.llnl.gov> Hey Sasha, Simple patch. Just change the command line option for consistency to the config file, variable name, etc. Alternately, we could rename everything else from ucast_dump_file to ucast_file. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-change-ucast_file-to-ucast_dump_file.patch Type: text/x-patch Size: 2276 bytes Desc: not available URL: From chu11 at llnl.gov Mon Jul 28 13:59:19 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 28 Jul 2008 13:59:19 -0700 Subject: [ofa-general] [OpenSM][Trivial] Change --ucast_file to --ucast_dump_file Message-ID: <1217278759.16508.360.camel@cardanus.llnl.gov> Hey Sasha, Simple patch. Just change the command line option for consistency to the config file, variable name, etc. Alternately, we could rename everything else from ucast_dump_file to ucast_file. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-change-ucast_file-to-ucast_dump_file.patch Type: text/x-patch Size: 2276 bytes Desc: not available URL: From sashak at voltaire.com Mon Jul 28 15:44:49 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 29 Jul 2008 01:44:49 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_lid_mgr.c: Cosmetic format change In-Reply-To: References: Message-ID: <20080728224449.GB5487@sashak.voltaire.com> On 15:54 Mon 28 Jul , Hal Rosenstock wrote: > opensm/osm_lid_mgr.c: Cosmetic format change > > Fix missing left brace introduced by previous patch > > Signed-off-by: Hal Rosenstock As far as I can see this brace was always missing. Applied anyway. Thanks. BTW please check your mailer - all recent patches are white space mangled, I need to apply the changes by hand. Sasha From sashak at voltaire.com Mon Jul 28 15:48:31 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 29 Jul 2008 01:48:31 +0300 Subject: [ofa-general] Re: [OpenSM][Trivial] Change --ucast_file to --ucast_dump_file In-Reply-To: <1217278759.16508.360.camel@cardanus.llnl.gov> References: <1217278759.16508.360.camel@cardanus.llnl.gov> Message-ID: <20080728224831.GC5487@sashak.voltaire.com> Hi Al, On 13:59 Mon 28 Jul , Al Chu wrote: > > Simple patch. Just change the command line option for consistency to > the config file, variable name, etc. > > Alternately, we could rename everything else from ucast_dump_file to > ucast_file. I think it would be better because this is not a dump file, but *input* LFTs file. Probably it is even better to rename it to 'lfts_file' or so. What do you think? Sasha From ralph.campbell at qlogic.com Mon Jul 28 16:53:54 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Mon, 28 Jul 2008 16:53:54 -0700 Subject: [ofa-general] Problem with ipath_dma.c and dma-mapping.h In-Reply-To: <200807250539.m6P5dWc04020@gandalf.iuzzolino.com> References: <200807250539.m6P5dWc04020@gandalf.iuzzolino.com> Message-ID: <1217289234.620.22.camel@chromite.mv.qlogic.com> After looking at this somewhat, I don't see what the bug is. The various OFED-1.4 files in: kernel_addons/backport/*/include/linux/dma-mapping.h either have or don't have #ifndef __x86_64__ depending on whether valid_dma_direction() is defined in include/asm-x86_64/dma-mapping.h or not. For the kernel-devel-2.6.18-1.2798.fc6.x86_64.rpm, include/asm-x86_64/dma-mapping.h does define valid_dma_direction() and OFED-1.4 kernel_addons/backport/backport/2.6.18_FC6/include/linux/dma-mapping.h does correctly ifndef __x86_64__ valid_dma_direction(). I don't have an FC6 system handy so I can't reproduce this. Are you sure that you aren't installing on a patched kernel? You can check by looking at: cd /lib/modules/`uname -r`/build/include grep valid_dma_direction asm-x86_64/dma-mapping.h linux/dma-mapping.h On Thu, 2008-07-24 at 23:39 -0600, Harold/Carlyn Iuzzolino wrote: > Dear Openfabrics, general at lists.openfabrics.org > > I am trying to compile OFED-1.4-20080724-0600.tgz, the latest-and-greatest > version as of today July 24. > > Fedora Core release 6 (Zod) > Kernel 2.6.18-1.2798.fc6xen on an x86_64 > > [root at elrond ~]# uname -a > Linux elrond 2.6.18-1.2798.fc6xen #1 SMP Mon Oct 16 14:59:01 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux > > It is a Single core 64 bit 2.4 GHz machine > > > I run the install.pl script with no changes and made these choices > OFED Distribution Software Installation Menu > 2) Install OFED Software > OFED Distribution Software Installation Menu > 3) All packages (all of Basic, HPC) > Please choose an implementation of MVAPICH2: > 1) OFA (IB and iWARP) > Enable ROMIO support [Y/n]: y > Enable shared library support [Y/n]: y > Enable Checkpoint-Restart support [y/N]: n > > > I have run into the following error: > > > /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c:58: error: implicit declaration of function 'valid_dma_direction' > > The code it is complaining about is > ipath_dma.c: BUG_ON(!valid_dma_direction(direction)); > > This subroutine valid_dma_direction is USED 6 times in one file ipath_dma.c > but isn't DEFINED anywhere. > > If I cd up 3 levels and search recursively for valid_dma_direction, > I find valid_dma_direction in 24 files, all dma-mapping.h for different > versions of the kernel_addons backports. > > dma-mapping.h has this kind of code: > #ifndef __x86_64__ > > /* x86_64 was the only architecture that had valid_dma_direction > * before 2.6.18 */ > > static inline int valid_dma_direction(int dma_direction) > { > return ((dma_direction == DMA_BIDIRECTIONAL) || > (dma_direction == DMA_TO_DEVICE) || > (dma_direction == DMA_FROM_DEVICE)); > } > #endif > > > I added the line > #include > to the file ipath_dma.c > (/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c) > > That didn't work. The reason is that since this is a 64 bit machine, the > definition: static inline int valid_dma_direction(int dma_direction) > isn't included because of the #ifndef __x86_64__ statement. > > So I went into the dma-mapping.h file and put > #undef __x86_64__ > before the ifndef __x86_64__ so that the definition of valid_dma_direction(int dma_direction) is included. > > > SUCCESS, sort of. That subroutine DID compile. BUT a file later on didn't > compile. > > BUT I shouldn't have to do fixes like that. And the question is, if > this is a 64 bit machine, and the dma-mapping.h defines valid_dma_direction > ONLY IF you are NOT compiling for a 64 bit machine, then the Openfabrics > files: ipath_dma.c and/or dma-mapping.h have some kind of mistake. > > Has anybody else run into this problem? Do you know what the correct code > is? Can you get Openfabrics to incorporate the fix into their code? > > Carlyn Iuzzolino > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From weiny2 at llnl.gov Mon Jul 28 08:48:21 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 28 Jul 2008 08:48:21 -0700 Subject: ***SPAM*** Re: [ofa-general] ***SPAM*** [PATCH] opensm/*/Makefile.am: remove explicit -lpthread and -ldl flags from Makefile.am In-Reply-To: <20080727184705.GV5067@sashak.voltaire.com> References: <20080727130916.GT5067@sashak.voltaire.com> <20080727184705.GV5067@sashak.voltaire.com> Message-ID: <20080728084821.48195028.weiny2@llnl.gov> Sasha, On Sun, 27 Jul 2008 21:47:05 +0300 Sasha Khapyorsky wrote: > > -lpthread and -ldl are standard things and can be generated by > /configure into LIBS variable and we don't need to specify this > explicitly in Makefiles. If so remove already not used lopensm_LDFLAGS > and osmtest_LDFLAGS variables too. > > Signed-off-by: Sasha Khapyorsky > > diff --git a/opensm/configure.in b/opensm/configure.in > index 7d04c14..7da932b 100644 > --- a/opensm/configure.in > +++ b/opensm/configure.in > @@ -38,6 +38,8 @@ fi > dnl Checks for libraries > AC_CHECK_LIB(pthread, pthread_mutex_init, [], > AC_MSG_ERROR([pthread_mutex_init() not found. libosmcomp requires libpthread.])) > +AC_CHECK_LIB(dl, dlopen, [], > + AC_MSG_ERROR([dlopen() not found. OpenSM requires libdl.])) > > dnl Checks for typedefs, structures, and compiler characteristics. > AC_C_CONST I think it is still valid to check for the dl lib. Ira From AHKumar at odu.edu Mon Jul 28 08:56:46 2008 From: AHKumar at odu.edu (Kumar, Amit H.) Date: Mon, 28 Jul 2008 11:56:46 -0400 Subject: ***SPAM*** RE: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created In-Reply-To: <488DEA6B.1060409@mellanox.co.il> References: , <488DEA6B.1060409@mellanox.co.il> Message-ID: <879DC79E-7C18-4C4D-A9FE-8A886C02BF71@mimectl> # rpm --showrc ARCHITECTURE AND OS: build arch : x86_64 compatible build archs: ia32e x86_64 noarch build os : Linux compatible build os's : linux install arch : ia32e install os : Linux compatible archs : ia32e x86_64 athlon noarch amd64 i686 i586 i486 i386 fat compatible os's : linux RPMRC VALUES: macrofiles : /usr/lib/rpm/macros:/usr/lib/rpm/x86_64-linux/macros:/usr/lib/rpm/redhat/macros:/etc/rpm/macros.*:/etc/rpm/macros:/etc/rpm/x86_64-linux/macros:~/.rpmmacros optflags : -O2 -g Features supported by rpmlib: rpmlib(VersionedDependencies) = 3.0.3-1 PreReq:, Provides:, and Obsoletes: dependencies support versions. rpmlib(CompressedFileNames) = 3.0.4-1 file name(s) stored as (dirName,baseName,dirIndex) tuple, not as path. rpmlib(PayloadIsBzip2) = 3.0.5-1 package payload can be compressed using bzip2. rpmlib(PayloadFilesHavePrefix) = 4.0-1 package payload file(s) have "./" prefix. rpmlib(ExplicitPackageProvide) = 4.0-1 package name-version-release is not implicitly provided. rpmlib(HeaderLoadSortsTags) = 4.0.1-1 header tags are always sorted after being loaded. rpmlib(ScriptletInterpreterArgs) = 4.0.3-1 the scriptlet interpreter can use arguments from header. rpmlib(PartialHardlinkSets) = 4.0.4-1 a hardlink file set may be installed without being complete. rpmlib(ConcurrentAccess) = 4.1-1 package scriptlets may access the rpm database while installing. rpmlib(BuiltinLuaScripts) = 4.2.2-1 internal support for lua scripts. ======================== -14: GNUconfigure(MCs:) CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS; LDFLAGS="${LDFLAGS:-%{-s:-s}}" ; export LDFLAGS; %{-C:_mydir="`pwd`"; %{-M: %{__mkdir} -p %{-C*};} cd %{-C*}} dirs="`find ${_mydir} -name configure.in -print`"; export dirs; for coin in `echo ${dirs}` do dr=`dirname ${coin}`; if test -f ${dr}/NO-AUTO-GEN; then : else macrodirs=`sed -n -e 's,AM_ACLOCAL_INCLUDE((.*)),1,gp' < ${coin}`; ( cd ${dr}; aclocalinclude="${ACLOCAL_FLAGS}"; for k in ${macrodirs}; do if test -d ${k}; then aclocalinclude="${aclocalinclude} -I ${k}"; ##else ## echo "**Warning**: No such directory `${k}'. Ignored." fi done if grep "^AM_GNU_GETTEXT" configure.in >/dev/null; then if grep "sed.*POTFILES" configure.in >/dev/null; then : do nothing -- we still have an old unmodified configure.in else test -r ${dr}/aclocal.m4 || touch ${dr}/aclocal.m4; echo "no" | gettextize --force --copy; test -r ${dr}/aclocal.m4 && %{__chmod} u+w ${dr}/aclocal.m4; fi fi if grep "^AM_PROG_LIBTOOL" configure.in >/dev/null; then %{__libtoolize} --force --copy; fi aclocal ${aclocalinclude}; if grep "^AM_CONFIG_HEADER" configure.in >/dev/null; then %{__autoheader}; fi echo "Running automake --gnu ${am_opt} ..."; %{__automake} --add-missing --gnu ${am_opt}; %{__autoconf}; ); fi done %{-C:${_mydir}}%{!-C:.}/configure %{_target_platform} --prefix=%{_prefix} --exec-prefix=%{_exec_prefix} --bindir=%{_bindir} --sbindir=%{_sbindir} --sysconfdir=%{_sysconfdir} --datadir=%{_datadir} --includedir=%{_includedir} --libdir=%{_libdir} --libexecdir=%{_libexecdir} --localstatedir=%{_localstatedir} --sharedstatedir=%{_sharedstatedir} --mandir=%{_mandir} --infodir=%{_infodir} %* ; %{-C:cd ${_mydir}; unset _mydir} -14: ___build_args -e -14: ___build_cmd %{?_sudo:%{_sudo} }%{?_remsh:%{_remsh} %{_remhost} }%{?_remsudo:%{_remsudo} }%{?_remchroot:%{_remchroot} %{_remroot} }%{___build_shell} %{___build_args} -14: ___build_post exit 0 -14: ___build_pre RPM_SOURCE_DIR="%{u2p:%{_sourcedir}}" RPM_BUILD_DIR="%{u2p:%{_builddir}}" RPM_OPT_FLAGS="%{optflags}" RPM_ARCH="%{_arch}" RPM_OS="%{_os}" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{name}" RPM_PACKAGE_VERSION="%{version}" RPM_PACKAGE_RELEASE="%{release}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE %{?buildroot:RPM_BUILD_ROOT="%{u2p:%{buildroot}}" export RPM_BUILD_ROOT} %{?_javaclasspath:CLASSPATH="%{_javaclasspath}" export CLASSPATH} %{verbose:set -x}%{!verbose:exec > /dev/null} umask 022 cd %{u2p:%{_builddir}} -14: ___build_shell %{?_buildshell:%{_buildshell}}%{!?_buildshell:/bin/sh} -14: ___build_template #!%{___build_shell} %{___build_pre} %{nil} -14: __aclocal aclocal -14: __ar ar -14: __arch_install_post %{nil} -14: __as as -14: __autoconf autoconf -14: __autoheader autoheader -14: __automake automake -14: __awk gawk -14: __bzip2 /usr/bin/bzip2 -14: __cat /bin/cat -14: __cc gcc -14: __check_files /usr/lib/rpm/check-files %{buildroot} -14: __chgrp /bin/chgrp -14: __chgrp_Rhf %{__chgrp} -Rhf -14: __chmod /bin/chmod -14: __chown /bin/chown -14: __chown_Rhf %{__chown} -Rhf -14: __cp /bin/cp -14: __cpio /bin/cpio -14: __cpp gcc -E -14: __cxx g++ -14: __dbi_btconfig btree %{__dbi_other} %{__dbi_perms} %{nil} -14: __dbi_btconfig_current %{__dbi_btconfig} -14: __dbi_btconfig_rebuild %{__dbi_btconfig} %{__dbi_rebuild} -14: __dbi_cdb create cdb mpool mp_mmapsize=16Mb mp_size=1Mb -14: __dbi_htconfig hash %{__dbi_other} %{__dbi_perms} %{nil} -14: __dbi_htconfig_current %{__dbi_htconfig} -14: __dbi_htconfig_rebuild %{__dbi_htconfig} %{__dbi_rebuild} -14: __dbi_other %{?_tmppath:tmpdir=%{_tmppath}} %{?__dbi_cdb} -14: __dbi_perms perms=0644 -14: __dbi_rebuild nofsync !log !txn !cdb -14: __dbi_transient %{__dbi_rebuild} temporary private -14: __debug_install_post /usr/lib/rpm/find-debuginfo.sh %{_builddir}/%{?buildsubdir} %{nil} -14: __file /usr/bin/file -14: __file_context_path /etc/selinux/%{__policy_tree}/contexts/files/file_contexts -14: __find_provides /usr/lib/rpm/redhat/find-provides -14: __find_requires /usr/lib/rpm/redhat/find-requires -14: __global_cflags -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -14: __gpg /usr/bin/gpg -14: __gpg_check_password_cmd %{__gpg} gpg --batch --no-verbose --passphrase-fd 3 -u "%{_gpg_name}" -so - -14: __gpg_sign_cmd %{__gpg} gpg --batch --no-verbose --no-armor --passphrase-fd 3 --no-secmem-warning -u "%{_gpg_name}" -sbo %{__signature_filename} %{__plaintext_filename} -14: __grep /bin/grep -14: __gzip /bin/gzip -14: __id /usr/bin/id -14: __id_u %{__id} -u -14: __install /usr/bin/install -14: __ld /usr/bin/ld -14: __libtoolize libtoolize -14: __ln_s ln -s -14: __make /usr/bin/make -14: __mkdir /bin/mkdir -14: __mkdir_p /bin/mkdir -p -14: __mono_provides /usr/lib/rpm/mono-find-provides %{_builddir}/%{?buildsubdir} %{buildroot} %{_libdir} -14: __mono_requires /usr/lib/rpm/mono-find-requires %{_builddir}/%{?buildsubdir} %{buildroot} %{_libdir} -14: __mv /bin/mv -14: __nm /usr/bin/nm -14: __objcopy /usr/bin/objcopy -14: __objdump /usr/bin/objdump -14: __os_install_post /usr/lib/rpm/redhat/brp-compress %{!?__debug_package:/usr/lib/rpm/redhat/brp-strip %{__strip}} /usr/lib/rpm/redhat/brp-strip-static-archive %{__strip} /usr/lib/rpm/redhat/brp-strip-comment-note %{__strip} %{__objdump} /usr/lib/rpm/brp-python-bytecompile /usr/lib/rpm/redhat/brp-java-repack-jars %{nil} -14: __patch /usr/bin/patch -14: __perl /usr/bin/perl -14: __perl_provides /usr/lib/rpm/perl.prov -14: __perl_requires /usr/lib/rpm/perl.req -14: __pgp /usr/bin/pgp -14: __pgp5_check_password_cmd %{__pgp} pgps +batchmode=on +verbose=0 +armor=off "%{_pgp_name}" -f -14: __pgp5_sign_cmd %{__pgp} pgps +batchmode=on +verbose=0 +armor=off "+myname=%{_pgp_name}" -b %{__plaintext_filename} -o %{__signature_filename} -14: __pgp_check_password_cmd %{__pgp} pgp +batchmode=on +verbose=0 "%{_pgp_name}" -sf -14: __pgp_sign_cmd %{__pgp} pgp +batchmode=on +verbose=0 +armor=off "+myname=%{_pgp_name}" -sb %{__plaintext_filename} %{__signature_filename} -14: __policy_tree %{expand:%%global __policy_tree %{lua: t="targeted" f = io.open("/etc/selinux/config") if f then for l in f:lines() do if "SELINUXTYPE=" == string.sub(l,0,12) then t=string.sub(l,13); end end f:close() end print (t) }}%{__policy_tree} -14: __prelink_undo_cmd /usr/sbin/prelink prelink -y library -14: __python /usr/bin/python -14: __python_provides /usr/lib/rpm/pythondeps.sh --provides -14: __python_requires /usr/lib/rpm/pythondeps.sh --requires -14: __ranlib ranlib -14: __remsh %{__rsh} -14: __rm /bin/rm -14: __rsh /usr/bin/rsh -14: __sed /bin/sed -14: __spec_build_args %{___build_args} -14: __spec_build_body %{___build_body} -14: __spec_build_cmd %{___build_cmd} -14: __spec_build_post %{___build_post} -14: __spec_build_pre %{___build_pre} -14: __spec_build_shell %{___build_shell} -14: __spec_build_template #!%{__spec_build_shell} %{__spec_build_pre} %{nil} -14: __spec_check_args %{___build_args} -14: __spec_check_body %{___build_body} -14: __spec_check_cmd %{___build_cmd} -14: __spec_check_post %{___build_post} -14: __spec_check_pre %{___build_pre} -14: __spec_check_shell %{___build_shell} -14: __spec_check_template #!%{__spec_check_shell} %{__spec_check_pre} %{nil} -14: __spec_clean_args %{___build_args} -14: __spec_clean_body %{___build_body} -14: __spec_clean_cmd %{___build_cmd} -14: __spec_clean_post %{___build_post} -14: __spec_clean_pre %{___build_pre} -14: __spec_clean_shell %{___build_shell} -14: __spec_clean_template #!%{__spec_clean_shell} %{__spec_clean_pre} %{nil} -14: __spec_install_args %{___build_args} -14: __spec_install_body %{___build_body} -14: __spec_install_cmd %{___build_cmd} -14: __spec_install_post %{?__debug_package:%{__debug_install_post}} %{__arch_install_post} %{__os_install_post} %{nil} -14: __spec_install_pre %{___build_pre} -14: __spec_install_shell %{___build_shell} -14: __spec_install_template #!%{__spec_install_shell} %{__spec_install_pre} %{nil} -14: __spec_prep_args %{___build_args} -14: __spec_prep_body %{___build_body} -14: __spec_prep_cmd %{___build_cmd} -14: __spec_prep_post %{___build_post} -14: __spec_prep_pre %{___build_pre} -14: __spec_prep_shell %{___build_shell} -14: __spec_prep_template #!%{__spec_prep_shell} %{__spec_prep_pre} %{nil} -14: __spec_rmbuild_args %{___build_args} -14: __spec_rmbuild_body %{___build_body} -14: __spec_rmbuild_cmd %{___build_cmd} -14: __spec_rmbuild_post %{___build_post} -14: __spec_rmbuild_pre %{___build_pre} -14: __spec_rmbuild_shell %{___build_shell} -14: __spec_rmbuild_template #!%{__spec_rmbuild_shell} %{__spec_rmbuild_pre} %{nil} -14: __ssh /usr/bin/ssh -14: __strip /usr/bin/strip -14: __tar /bin/tar -14: __unzip /usr/bin/unzip -14: __vsflags 0xc0c00 -14: _arch x86_64 -14: _autorelocate_dcolor 0 -14: _autorelocate_path %{nil} -14: _bhA RPMS -14: _bhN @(SRPMS|i386|alpha|sparc|s390|ia64) -14: _bhVR RedHat -14: _bhcoll @(7.3|7.2|7.1|7.1sbe|7.1k|7.0|7.01j|7.0j|7.0sbe|7.0tc|6.2|6.2ha|6.2ee|6.1|6.0|5.2|5.1|5.0) -14: _bhpath file://localhost/mnt/dist -14: _bindir %{_exec_prefix}/bin -14: _build %{_host} -14: _build_alias %{_host_alias} -14: _build_arch x86_64 -14: _build_cpu %{_host_cpu} -14: _build_file_context_path %{nil} -14: _build_name_fmt %%{ARCH}/%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm -14: _build_os %{_host_os} -14: _build_vendor %{_host_vendor} -14: _builddir %{_topdir}/BUILD -14: _buildshell /bin/sh -14: _bzip2bin %{__bzip2} -14: _cache_dbpath /var/spool/up2date/cache -14: _datadir %{_prefix}/share -14: _dbapi 3 -14: _dbapi_rebuild 3 -14: _dbi_btconfig %{?_rpmdb_rebuild:%{__dbi_btconfig_rebuild}} %{!?_rpmdb_rebuild:%{__dbi_btconfig_current}} %{nil} -14: _dbi_config %{_dbi_htconfig} -14: _dbi_config_Depends %{_dbi_htconfig} temporary private -14: _dbi_config_Dirnames %{_dbi_btconfig} -14: _dbi_config_Installtid %{_dbi_btconfig} -14: _dbi_config_Packages %{_dbi_htconfig} lockdbfd -14: _dbi_config_Provideversion %{_dbi_btconfig} -14: _dbi_config_Removetid %{_dbi_btconfig} -14: _dbi_config_Requireversion %{_dbi_btconfig} -14: _dbi_htconfig %{?_rpmdb_rebuild:%{__dbi_htconfig_rebuild}} %{!?_rpmdb_rebuild:%{__dbi_htconfig_current}} %{nil} -14: _dbi_tags Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Depends:Pubkeys -14: _dbpath %{_var}/lib/rpm -14: _dbpath_rebuild %{_dbpath} -14: _defaultdocdir %{_usr}/share/doc -14: _dependency_whiteout libtermcap>bash modutils>vixie-cron ypbind>yp-tools ghostscript-fonts>ghostscript %{?_dependency_whiteout_fc3} %{?_dependency_whiteout_fc2} %{?_dependency_whiteout_fc1} %{?_dependency_whiteout_9} %{?_dependency_whiteout_8_0} %{?_dependency_whiteout_7_2} %{?_dependency_whiteout_7_1} %{?_dependency_whiteout_7_0} %{?_dependency_whiteout_6_2} %{?_dependency_whiteout_6_1} %{?_dependency_whiteout_5_2} %{nil} -14: _dependency_whiteout_7_0 pango-gtkbeta-devel>pango-gtkbeta XFree86>Mesa compat-glibc>db2 compat-glibc>db1 pam>initscripts initscripts>sysklogd -14: _dependency_whiteout_7_1 arts>kdelibs-sound -14: _dependency_whiteout_7_2 libgnomeprint15>gnome-print nautilus>nautilus-mozilla tcl>postgresql-tcl -14: _dependency_whiteout_8_0 mysql>perl-DBD-MySQL perl>perl-Filter perl>mrtg perl>mod_perl perl-Date-Calc>perl-Bit-Vector -14: _dependency_whiteout_fc3 coreutils>pam nautilus>nautilus-cd-burner aspell>aspell-en kernel>initscripts kernel-smp>initscripts xorg-x11-libs>xorg-x11-Mesa-libGL openldap>cyrus-sasl-md5 openldap>cyrus-sasl openjade>docbook-dtds gtk+>gdk-pixbuf xorg-x11>xinitrc gnome-python2>gnome-python2-bonobo httpd-suexec>httpd xemacs-sumo>apel-xemacs php>php-pear openoffice.org-libs>openoffice.org -14: _desktopdir %{_datadir}/applications -14: _enable_debug_packages 1 -14: _exec_prefix %{_prefix} -14: _filter_GLIBC_PRIVATE 0 -14: _fixgroup [ `%{__id_u}` = '0' ] && %{__chgrp_Rhf} root -14: _fixowner [ `%{__id_u}` = '0' ] && %{__chown_Rhf} root -14: _fixperms %{__chmod} -Rf a+rX,u+w,g-w,o-w -14: _gcj_support 1 -14: _gnu -gnu -14: _gzipbin %{__gzip} -14: _hkp_keyserver %{nil} -14: _host x86_64-redhat-linux-gnu -14: _host_alias %{nil} -14: _host_cpu x86_64 -14: _host_os linux-gnu -14: _host_vendor redhat -14: _i18ndomains redhat-dist -14: _icons16dir %{_iconsbasedir}/16x16/apps -14: _icons192dir %{_iconsbasedir}/192x192/apps -14: _icons22dir %{_iconsbasedir}/22x22/apps -14: _icons48dir %{_iconsbasedir}/48x48/apps -14: _icons64dir %{_iconsbasedir}/64x64/apps -14: _icons96dir %{_iconsbasedir}/96x96/apps -14: _iconsbasedir %{_datadir}/icons/%{_iconstheme} -14: _iconsdir %{_datadir}/icons -14: _iconsscaldir %{_iconsbasedir}/scalable/apps -14: _iconstheme hicolor -14: _includedir %{_prefix}/include -14: _infodir /usr/share/info -14: _initrddir %{_sysconfdir}/rc.d/init.d -14: _install_file_context_path %{__file_context_path} -14: _install_langs all -14: _install_script_path /sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin -14: _instchangelog 5 -14: _javadir %{_datadir}/java -14: _javadocdir %{_datadir}/javadoc -14: _jnidir %{_prefix}/lib/java -14: _jvmcommondatadir %{_datadir}/jvm-commmon -14: _jvmcommonlibdir %{_prefix}/lib/jvm-commmon -14: _jvmcommonsysconfdir %{_sysconfdir}/jvm-commmon -14: _jvmdatadir %{_datadir}/jvm -14: _jvmdir %{_prefix}/lib/jvm -14: _jvmjardir %{_prefix}/lib/jvm-exports -14: _jvmlibdir %{_prefix}/lib/jvm -14: _jvmprivdir %{_prefix}/lib/jvm-private -14: _jvmsysconfdir %{_sysconfdir}/jvm -14: _lib lib64 -14: _libdir %{_exec_prefix}/%{_lib} -14: _libexecdir %{_exec_prefix}/libexec -14: _liconsdir %{_datadir}/icons/large -14: _localstatedir /var -14: _mandir /usr/share/man -14: _mavendepmapdir /etc/maven -14: _mavendepmapfragdir /etc/maven/fragments -14: _menudir %{_prefix}/lib/menu -14: _miconsdir %{_datadir}/icons/mini -14: _missing_doc_files_terminate_build 0 -14: _multilibno 2 -14: _multilibpatt (/%{_lib}|/usr/%{_lib}(|/gconv)|/usr/local/%{_lib}|/usr/X11R6/%{_lib}|/opt/%{_lib})/[^/]*\.([oa]|la|so[0-9.]*)$ -14: _oldincludedir /usr/include -14: _os linux -14: _package_version 30005 -14: _pgpbin %{__pgp} -15: _preScriptEnvironment RPM_SOURCE_DIR="%{_sourcedir}" RPM_BUILD_DIR="%{_builddir}" RPM_OPT_FLAGS="%{optflags}" RPM_ARCH="%{_arch}" RPM_OS="%{_os}" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{name}" RPM_PACKAGE_VERSION="%{version}" RPM_PACKAGE_RELEASE="%{release}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE %{?buildroot:RPM_BUILD_ROOT="%{buildroot}" export RPM_BUILD_ROOT } -14: _prefix /usr -14: _query_all_fmt %%{name}-%%{version}-%%{release} -14: _query_selector_match default -14: _repackage_all_erasures 0 -14: _repackage_dir /var/spool/repackage -14: _repackage_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm -14: _repackage_root %{nil} -14: _rollback_transaction_on_failure 0 -14: _rpmdir %{_topdir}/RPMS -14: _rpmfilename %{_build_name_fmt} -14: _rpmlock_path %{_dbpath}/__db.000 -14: _sbindir %{_exec_prefix}/sbin -14: _sharedstatedir %{_prefix}/com -14: _signature none -14: _smp_mflags %([ -z "$RPM_BUILD_NCPUS" ] \ && RPM_BUILD_NCPUS="`/usr/bin/getconf _NPROCESSORS_ONLN`"; \ [ "$RPM_BUILD_NCPUS" -gt 1 ] && echo "-j$RPM_BUILD_NCPUS") -14: _sourcedir %{_topdir}/SOURCES -14: _specdir %{_topdir}/SPECS -14: _srcrpmdir %{_topdir}/SRPMS -14: _sysconfdir /etc -11: _target x86_64-linux -14: _target_alias %{_host_alias} -11= _target_cpu x86_64 -11= _target_os linux -14: _target_platform %{_target_cpu}-%{_vendor}-%{_target_os}%{?_gnu} -14: _target_vendor %{_host_vendor} -14: _tmppath %{_var}/tmp -14: _topdir %{_usrsrc}/redhat -14: _transaction_color 3 -14: _unpackaged_files_terminate_build 0 -14: _unzipbin %{__unzip} -14: _use_internal_dependency_generator 1 -14: _usr /usr -14: _usrsrc %{_usr}/src -14: _var /var -14: _vendor redhat -14: _verify_file_context_path %{__file_context_path} -14: _vsflags_build %{__vsflags} -14: _vsflags_erase %{__vsflags} -14: _vsflags_install %{__vsflags} -14: _vsflags_query %{__vsflags} -14: _vsflags_rebuilddb %{__vsflags} -14: _vsflags_up2date %{__vsflags} -14: _vsflags_verify %{__vsflags} -14: _without_check 0 -14: add_jvm_extension JAVA_LIBDIR=%{buildroot}/%{_javadir} %{_bindir}/jvmjar -l -14: add_to_maven_depmap install -dm 755 $RPM_BUILD_ROOT/%{_mavendepmapfragdir} cat >>$RPM_BUILD_ROOT/%{_mavendepmapfragdir}/%{name}<< EOF %1 %2 %3 %4 %5 %3 EOF %{nil} -14: ant JAVA_HOME=%{java_home} ant -14: bcond_with %{expand:%%{?_with_%{1}:%%global with_%{1} 1}} -14: bcond_without %{expand:%%{!?_without_%{1}:%%global with_%{1} 1}} -14: build %%build LANG=C export LANG unset DISPLAY %{nil} -14: configure CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS ; CXXFLAGS="${CXXFLAGS:-%optflags}" ; export CXXFLAGS ; FFLAGS="${FFLAGS:-%optflags}" ; export FFLAGS ; for i in $(find . -name config.guess -o -name config.sub) ; do [ -f /usr/lib/rpm/redhat/$(basename $i) ] && %{__rm} -f $i && %{__cp} -fv /usr/lib/rpm/redhat/$(basename $i) $i ; done ; ./configure --build=%{_build} --host=%{_host} \ --target=%{_target_platform} \ --program-prefix=%{?_program_prefix} \ --prefix=%{_prefix} \ --exec-prefix=%{_exec_prefix} \ --bindir=%{_bindir} \ --sbindir=%{_sbindir} \ --sysconfdir=%{_sysconfdir} \ --datadir=%{_datadir} \ --includedir=%{_includedir} \ --libdir=%{_libdir} \ --libexecdir=%{_libexecdir} \ --localstatedir=%{_localstatedir} \ --sharedstatedir=%{_sharedstatedir} \ --mandir=%{_mandir} \ --infodir=%{_infodir} -14: debug_package %{nil} -14: defined %{expand:%%{?%{1}:1}%%{!?%{1}:0}} -14: distribution Rocks -14: find_lang /usr/lib/rpm/redhat/find-lang.sh %{buildroot} -14: install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}} %%install LANG=C export LANG unset DISPLAY %{nil} -14: ix86 i386 i486 i586 i686 pentium3 pentium4 athlon -14: jar %{java_home}/bin/jar -14: java %(. %{_javadir}-utils/java-functions; set_javacmd; echo $JAVACMD) -14: java_home %(. %{_javadir}-utils/java-functions; set_jvm; echo $JAVA_HOME) -14: javac %{java_home}/bin/javac -14: javadoc %{java_home}/bin/javadoc -14: jpackage_script install -d -m 755 $RPM_BUILD_ROOT%{_bindir} cat > $RPM_BUILD_ROOT%{_bindir}/%5 << EOF #!/bin/sh # # %{name} script # JPackage Project # Source functions library . %{_javadir}-utils/java-functions # Source system prefs if [ -f %{_sysconfdir}/java/%{name}.conf ] ; then . %{_sysconfdir}/java/%{name}.conf fi # Source user prefs if [ -f \$HOME/.%{name}rc ] ; then . \$HOME/.%{name}rc fi # Configuration MAIN_CLASS=%1 BASE_FLAGS=%2 BASE_OPTIONS=%3 BASE_JARS="%(echo %4 | sed -e 's,:, ,g')" # Set parameters set_jvm set_classpath \$BASE_JARS set_flags \$BASE_FLAGS set_options \$BASE_OPTIONS # Let's start run "\$@" EOF -14: kernel_module_package(n:v:r:s:f:xp:) %{expand:%( %define kmodtool %{-s*}%{!-s:/usr/lib/rpm/redhat/kmodtool} %define kmp_version %{-v*}%{!-v:%{version}} %define kmp_release %{-r*}%{!-r:%{release}} %define latest_kernel %(rpm -q --qf '%{VERSION}-%{RELEASE}\\n' `rpm -q kernel-devel | /usr/lib/rpm/redhat/rpmsort -r | head -n 1` | head -n 1) %{!?kernel_version:%{expand:%%global kernel_version %{latest_kernel}}} %global kverrel %(%{kmodtool} verrel %{?kernel_version} 2>/dev/null) flavors="default" if [ "i686" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) kdver=$(rpm -q kernel-kdump-devel-%{kverrel}|head -n 1) paever=$(rpm -q kernel-PAE-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi if [ "kernel-kdump-devel-%{kverrel}" == "$kdver" ] then flavors="$flavors kdump" fi if [ "kernel-PAE-devel-%{kverrel}" == "$paever" ] then flavors="$flavors PAE" fi fi if [ "x86_64" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi fi if [ "ppc64" == "%{_target_cpu}" ] || [ "ppc64iseries" == "%{_target_cpu}" ] then kdver=$(rpm -q kernel-kdump-devel-%{kverrel}|head -n 1) if [ "kernel-kdump-devel-%{kverrel}" == "$kdver" ] then flavors="$flavors kdump" fi fi if [ "ia64" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi fi if [ -z "%*" ]; then flavors_to_build=$flavors elif [ -z "%{-x}" ]; then flavors_to_build="%*" else flavors_to_build=" $flavors " echo "[$flavors_to_build]" >/tmp/tmp.txt for i in %* do flavors_to_build=${flavors_to_build//$i /} done fi echo "%%global flavors_to_build ${flavors_to_build:-%%nil}" echo "%%global kernel_source() /usr/src/kernels/%kverrel-\$([ %%%%{1} = default ] || echo "%%%%{1}-")%_target_cpu" if [ ! -z "%{-f*}" ] then filelist="%{-f*}" fi if [ ! -z "%{-p*}" ] then preamble="%{-p*}" fi if [ -z "%{kmodtool_generate_buildreqs}" ] then nobuildreqs="yes" fi kmp_override_filelist="$filelist" kmp_override_preamble="$preamble" kmp_nobuildreqs="$buildreqs" %{kmodtool} rpmtemplate_kmp %{-n*}%{!-n:%name} %{kverrel} $flavors_to_build 2>/dev/null )} -14: kernel_module_package_buildreqs %global kmodtool_generate_buildreqs 1 kernel-devel -14: makeinstall %{__make} \ prefix=%{?buildroot:%{buildroot}}%{_prefix} \ exec_prefix=%{?buildroot:%{buildroot}}%{_exec_prefix} \ bindir=%{?buildroot:%{buildroot}}%{_bindir} \ sbindir=%{?buildroot:%{buildroot}}%{_sbindir} \ sysconfdir=%{?buildroot:%{buildroot}}%{_sysconfdir} \ datadir=%{?buildroot:%{buildroot}}%{_datadir} \ includedir=%{?buildroot:%{buildroot}}%{_includedir} \ libdir=%{?buildroot:%{buildroot}}%{_libdir} \ libexecdir=%{?buildroot:%{buildroot}}%{_libexecdir} \ localstatedir=%{?buildroot:%{buildroot}}%{_localstatedir} \ sharedstatedir=%{?buildroot:%{buildroot}}%{_sharedstatedir} \ mandir=%{?buildroot:%{buildroot}}%{_mandir} \ infodir=%{?buildroot:%{buildroot}}%{_infodir} \ install -14: mdkicons install -D -m 644 %1-48.png $RPM_BUILD_ROOT%{_liconsdir}/%1.png install -D -m 644 %1-32.png $RPM_BUILD_ROOT%{_iconsdir}/%1.png install -D -m 644 %1-16.png $RPM_BUILD_ROOT%{_miconsdir}/%1.png -14: mdkmenu install -d -m 755 $RPM_BUILD_ROOT%{_menudir} cat >$RPM_BUILD_ROOT%{_menudir}/%1 <$RPM_BUILD_ROOT%{_sysconfdir}/X11/applnk/%4/%1.desktop <\n" > %{_mavendepmapdir}/maven2-depmap.xml if [ -d %{_mavendepmapfragdir} ] && [ -n "`find %{_mavendepmapfragdir} -type f`" ]; then cat %{_mavendepmapfragdir}/* >> %{_mavendepmapdir}/maven2-depmap.xml fi echo -e "\n" >> %{_mavendepmapdir}/maven2-depmap.xml -14: with %{expand:%%{?with_%{1}:1}%%{!?with_%{1}:0}} -14: without %{expand:%%{?with_%{1}:0}%%{!?with_%{1}:1}} ======================== active 327 empty 0 Thank you, Amit ________________________________ From: general-bounces at lists.openfabrics.org [general-bounces at lists.openfabrics.org] On Behalf Of Vladimir Sokolovsky [vlad at mellanox.co.il] Sent: Monday, July 28, 2008 11:48 AM To: Kumar, Amit H. Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created Kumar, Amit H. wrote: > Hello ... > > Installing OFED 1.3.1 for the first time. I get the following message on > building RPMS and then installation stops. > > It creates Kernel-ib, kernel-ib-devel and ib-bonding rpms and then fails > at the following. > > Running rpmbuild --rebuild --define '_topdir /var/tmp/OFED_topdir' > --define 'KVERSION 2.6.18-53.1.21.el5' --define '_release > 2.6.18_53.1.21.el5' --define '_prefix /usr' > /share/apps/OFED-1.3.1/SRPMS/ib-bonding-0.9.0-25.src.rpm > ib-bonding-debuginfo was not created > > I then customised the installation to exclude ib-bonding-debuginfo, > installation continued and failed to build any debuginfo RPMS. Finally I > could complete the installation without any debuginfo libraries. > > Any idea what could I be doing wrong ??? > > Thank you, > Amit > Can you please send the output of 'rpm --showrc'? Regards, Vladimir _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -------------- next part -------------- An HTML attachment was scrubbed... URL: From AHKumar at odu.edu Mon Jul 28 09:18:49 2008 From: AHKumar at odu.edu (Kumar, Amit H.) Date: Mon, 28 Jul 2008 12:18:49 -0400 Subject: ***SPAM*** RE: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created In-Reply-To: <879DC79E-7C18-4C4D-A9FE-8A886C02BF71@mimectl> References: , <488DEA6B.1060409@mellanox.co.il>, <879DC79E-7C18-4C4D-A9FE-8A886C02BF71@mimectl> Message-ID: # rpm --showrc ARCHITECTURE AND OS: build arch : x86_64 compatible build archs: ia32e x86_64 noarch build os : Linux compatible build os's : linux install arch : ia32e install os : Linux compatible archs : ia32e x86_64 athlon noarch amd64 i686 i586 i486 i386 fat compatible os's : linux RPMRC VALUES: macrofiles : /usr/lib/rpm/macros:/usr/lib/rpm/x86_64-linux/macros:/usr/lib/rpm/redhat/macros:/etc/rpm/macros.*:/etc/rpm/macros:/etc/rpm/x86_64-linux/macros:~/.rpmmacros optflags : -O2 -g Features supported by rpmlib: rpmlib(VersionedDependencies) = 3.0.3-1 PreReq:, Provides:, and Obsoletes: dependencies support versions. rpmlib(CompressedFileNames) = 3.0.4-1 file name(s) stored as (dirName,baseName,dirIndex) tuple, not as path. rpmlib(PayloadIsBzip2) = 3.0.5-1 package payload can be compressed using bzip2. rpmlib(PayloadFilesHavePrefix) = 4.0-1 package payload file(s) have "./" prefix. rpmlib(ExplicitPackageProvide) = 4.0-1 package name-version-release is not implicitly provided. rpmlib(HeaderLoadSortsTags) = 4.0.1-1 header tags are always sorted after being loaded. rpmlib(ScriptletInterpreterArgs) = 4.0.3-1 the scriptlet interpreter can use arguments from header. rpmlib(PartialHardlinkSets) = 4.0.4-1 a hardlink file set may be installed without being complete. rpmlib(ConcurrentAccess) = 4.1-1 package scriptlets may access the rpm database while installing. rpmlib(BuiltinLuaScripts) = 4.2.2-1 internal support for lua scripts. ======================== -14: GNUconfigure(MCs:) CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS; LDFLAGS="${LDFLAGS:-%{-s:-s}}" ; export LDFLAGS; %{-C:_mydir="`pwd`"; %{-M: %{__mkdir} -p %{-C*};} cd %{-C*}} dirs="`find ${_mydir} -name configure.in -print`"; export dirs; for coin in `echo ${dirs}` do dr=`dirname ${coin}`; if test -f ${dr}/NO-AUTO-GEN; then : else macrodirs=`sed -n -e 's,AM_ACLOCAL_INCLUDE((.*)),1,gp' < ${coin}`; ( cd ${dr}; aclocalinclude="${ACLOCAL_FLAGS}"; for k in ${macrodirs}; do if test -d ${k}; then aclocalinclude="${aclocalinclude} -I ${k}"; ##else ## echo "**Warning**: No such directory `${k}'. Ignored." fi done if grep "^AM_GNU_GETTEXT" configure.in >/dev/null; then if grep "sed.*POTFILES" configure.in >/dev/null; then : do nothing -- we still have an old unmodified configure.in else test -r ${dr}/aclocal.m4 || touch ${dr}/aclocal.m4; echo "no" | gettextize --force --copy; test -r ${dr}/aclocal.m4 && %{__chmod} u+w ${dr}/aclocal.m4; fi fi if grep "^AM_PROG_LIBTOOL" configure.in >/dev/null; then %{__libtoolize} --force --copy; fi aclocal ${aclocalinclude}; if grep "^AM_CONFIG_HEADER" configure.in >/dev/null; then %{__autoheader}; fi echo "Running automake --gnu ${am_opt} ..."; %{__automake} --add-missing --gnu ${am_opt}; %{__autoconf}; ); fi done %{-C:${_mydir}}%{!-C:.}/configure %{_target_platform} --prefix=%{_prefix} --exec-prefix=%{_exec_prefix} --bindir=%{_bindir} --sbindir=%{_sbindir} --sysconfdir=%{_sysconfdir} --datadir=%{_datadir} --includedir=%{_includedir} --libdir=%{_libdir} --libexecdir=%{_libexecdir} --localstatedir=%{_localstatedir} --sharedstatedir=%{_sharedstatedir} --mandir=%{_mandir} --infodir=%{_infodir} %* ; %{-C:cd ${_mydir}; unset _mydir} -14: ___build_args -e -14: ___build_cmd %{?_sudo:%{_sudo} }%{?_remsh:%{_remsh} %{_remhost} }%{?_remsudo:%{_remsudo} }%{?_remchroot:%{_remchroot} %{_remroot} }%{___build_shell} %{___build_args} -14: ___build_post exit 0 -14: ___build_pre RPM_SOURCE_DIR="%{u2p:%{_sourcedir}}" RPM_BUILD_DIR="%{u2p:%{_builddir}}" RPM_OPT_FLAGS="%{optflags}" RPM_ARCH="%{_arch}" RPM_OS="%{_os}" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{name}" RPM_PACKAGE_VERSION="%{version}" RPM_PACKAGE_RELEASE="%{release}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE %{?buildroot:RPM_BUILD_ROOT="%{u2p:%{buildroot}}" export RPM_BUILD_ROOT} %{?_javaclasspath:CLASSPATH="%{_javaclasspath}" export CLASSPATH} %{verbose:set -x}%{!verbose:exec > /dev/null} umask 022 cd %{u2p:%{_builddir}} -14: ___build_shell %{?_buildshell:%{_buildshell}}%{!?_buildshell:/bin/sh} -14: ___build_template #!%{___build_shell} %{___build_pre} %{nil} -14: __aclocal aclocal -14: __ar ar -14: __arch_install_post %{nil} -14: __as as -14: __autoconf autoconf -14: __autoheader autoheader -14: __automake automake -14: __awk gawk -14: __bzip2 /usr/bin/bzip2 -14: __cat /bin/cat -14: __cc gcc -14: __check_files /usr/lib/rpm/check-files %{buildroot} -14: __chgrp /bin/chgrp -14: __chgrp_Rhf %{__chgrp} -Rhf -14: __chmod /bin/chmod -14: __chown /bin/chown -14: __chown_Rhf %{__chown} -Rhf -14: __cp /bin/cp -14: __cpio /bin/cpio -14: __cpp gcc -E -14: __cxx g++ -14: __dbi_btconfig btree %{__dbi_other} %{__dbi_perms} %{nil} -14: __dbi_btconfig_current %{__dbi_btconfig} -14: __dbi_btconfig_rebuild %{__dbi_btconfig} %{__dbi_rebuild} -14: __dbi_cdb create cdb mpool mp_mmapsize=16Mb mp_size=1Mb -14: __dbi_htconfig hash %{__dbi_other} %{__dbi_perms} %{nil} -14: __dbi_htconfig_current %{__dbi_htconfig} -14: __dbi_htconfig_rebuild %{__dbi_htconfig} %{__dbi_rebuild} -14: __dbi_other %{?_tmppath:tmpdir=%{_tmppath}} %{?__dbi_cdb} -14: __dbi_perms perms=0644 -14: __dbi_rebuild nofsync !log !txn !cdb -14: __dbi_transient %{__dbi_rebuild} temporary private -14: __debug_install_post /usr/lib/rpm/find-debuginfo.sh %{_builddir}/%{?buildsubdir} %{nil} -14: __file /usr/bin/file -14: __file_context_path /etc/selinux/%{__policy_tree}/contexts/files/file_contexts -14: __find_provides /usr/lib/rpm/redhat/find-provides -14: __find_requires /usr/lib/rpm/redhat/find-requires -14: __global_cflags -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -14: __gpg /usr/bin/gpg -14: __gpg_check_password_cmd %{__gpg} gpg --batch --no-verbose --passphrase-fd 3 -u "%{_gpg_name}" -so - -14: __gpg_sign_cmd %{__gpg} gpg --batch --no-verbose --no-armor --passphrase-fd 3 --no-secmem-warning -u "%{_gpg_name}" -sbo %{__signature_filename} %{__plaintext_filename} -14: __grep /bin/grep -14: __gzip /bin/gzip -14: __id /usr/bin/id -14: __id_u %{__id} -u -14: __install /usr/bin/install -14: __ld /usr/bin/ld -14: __libtoolize libtoolize -14: __ln_s ln -s -14: __make /usr/bin/make -14: __mkdir /bin/mkdir -14: __mkdir_p /bin/mkdir -p -14: __mono_provides /usr/lib/rpm/mono-find-provides %{_builddir}/%{?buildsubdir} %{buildroot} %{_libdir} -14: __mono_requires /usr/lib/rpm/mono-find-requires %{_builddir}/%{?buildsubdir} %{buildroot} %{_libdir} -14: __mv /bin/mv -14: __nm /usr/bin/nm -14: __objcopy /usr/bin/objcopy -14: __objdump /usr/bin/objdump -14: __os_install_post /usr/lib/rpm/redhat/brp-compress %{!?__debug_package:/usr/lib/rpm/redhat/brp-strip %{__strip}} /usr/lib/rpm/redhat/brp-strip-static-archive %{__strip} /usr/lib/rpm/redhat/brp-strip-comment-note %{__strip} %{__objdump} /usr/lib/rpm/brp-python-bytecompile /usr/lib/rpm/redhat/brp-java-repack-jars %{nil} -14: __patch /usr/bin/patch -14: __perl /usr/bin/perl -14: __perl_provides /usr/lib/rpm/perl.prov -14: __perl_requires /usr/lib/rpm/perl.req -14: __pgp /usr/bin/pgp -14: __pgp5_check_password_cmd %{__pgp} pgps +batchmode=on +verbose=0 +armor=off "%{_pgp_name}" -f -14: __pgp5_sign_cmd %{__pgp} pgps +batchmode=on +verbose=0 +armor=off "+myname=%{_pgp_name}" -b %{__plaintext_filename} -o %{__signature_filename} -14: __pgp_check_password_cmd %{__pgp} pgp +batchmode=on +verbose=0 "%{_pgp_name}" -sf -14: __pgp_sign_cmd %{__pgp} pgp +batchmode=on +verbose=0 +armor=off "+myname=%{_pgp_name}" -sb %{__plaintext_filename} %{__signature_filename} -14: __policy_tree %{expand:%%global __policy_tree %{lua: t="targeted" f = io.open("/etc/selinux/config") if f then for l in f:lines() do if "SELINUXTYPE=" == string.sub(l,0,12) then t=string.sub(l,13); end end f:close() end print (t) }}%{__policy_tree} -14: __prelink_undo_cmd /usr/sbin/prelink prelink -y library -14: __python /usr/bin/python -14: __python_provides /usr/lib/rpm/pythondeps.sh --provides -14: __python_requires /usr/lib/rpm/pythondeps.sh --requires -14: __ranlib ranlib -14: __remsh %{__rsh} -14: __rm /bin/rm -14: __rsh /usr/bin/rsh -14: __sed /bin/sed -14: __spec_build_args %{___build_args} -14: __spec_build_body %{___build_body} -14: __spec_build_cmd %{___build_cmd} -14: __spec_build_post %{___build_post} -14: __spec_build_pre %{___build_pre} -14: __spec_build_shell %{___build_shell} -14: __spec_build_template #!%{__spec_build_shell} %{__spec_build_pre} %{nil} -14: __spec_check_args %{___build_args} -14: __spec_check_body %{___build_body} -14: __spec_check_cmd %{___build_cmd} -14: __spec_check_post %{___build_post} -14: __spec_check_pre %{___build_pre} -14: __spec_check_shell %{___build_shell} -14: __spec_check_template #!%{__spec_check_shell} %{__spec_check_pre} %{nil} -14: __spec_clean_args %{___build_args} -14: __spec_clean_body %{___build_body} -14: __spec_clean_cmd %{___build_cmd} -14: __spec_clean_post %{___build_post} -14: __spec_clean_pre %{___build_pre} -14: __spec_clean_shell %{___build_shell} -14: __spec_clean_template #!%{__spec_clean_shell} %{__spec_clean_pre} %{nil} -14: __spec_install_args %{___build_args} -14: __spec_install_body %{___build_body} -14: __spec_install_cmd %{___build_cmd} -14: __spec_install_post %{?__debug_package:%{__debug_install_post}} %{__arch_install_post} %{__os_install_post} %{nil} -14: __spec_install_pre %{___build_pre} -14: __spec_install_shell %{___build_shell} -14: __spec_install_template #!%{__spec_install_shell} %{__spec_install_pre} %{nil} -14: __spec_prep_args %{___build_args} -14: __spec_prep_body %{___build_body} -14: __spec_prep_cmd %{___build_cmd} -14: __spec_prep_post %{___build_post} -14: __spec_prep_pre %{___build_pre} -14: __spec_prep_shell %{___build_shell} -14: __spec_prep_template #!%{__spec_prep_shell} %{__spec_prep_pre} %{nil} -14: __spec_rmbuild_args %{___build_args} -14: __spec_rmbuild_body %{___build_body} -14: __spec_rmbuild_cmd %{___build_cmd} -14: __spec_rmbuild_post %{___build_post} -14: __spec_rmbuild_pre %{___build_pre} -14: __spec_rmbuild_shell %{___build_shell} -14: __spec_rmbuild_template #!%{__spec_rmbuild_shell} %{__spec_rmbuild_pre} %{nil} -14: __ssh /usr/bin/ssh -14: __strip /usr/bin/strip -14: __tar /bin/tar -14: __unzip /usr/bin/unzip -14: __vsflags 0xc0c00 -14: _arch x86_64 -14: _autorelocate_dcolor 0 -14: _autorelocate_path %{nil} -14: _bhA RPMS -14: _bhN @(SRPMS|i386|alpha|sparc|s390|ia64) -14: _bhVR RedHat -14: _bhcoll @(7.3|7.2|7.1|7.1sbe|7.1k|7.0|7.01j|7.0j|7.0sbe|7.0tc|6.2|6.2ha|6.2ee|6.1|6.0|5.2|5.1|5.0) -14: _bhpath file://localhost/mnt/dist -14: _bindir %{_exec_prefix}/bin -14: _build %{_host} -14: _build_alias %{_host_alias} -14: _build_arch x86_64 -14: _build_cpu %{_host_cpu} -14: _build_file_context_path %{nil} -14: _build_name_fmt %%{ARCH}/%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm -14: _build_os %{_host_os} -14: _build_vendor %{_host_vendor} -14: _builddir %{_topdir}/BUILD -14: _buildshell /bin/sh -14: _bzip2bin %{__bzip2} -14: _cache_dbpath /var/spool/up2date/cache -14: _datadir %{_prefix}/share -14: _dbapi 3 -14: _dbapi_rebuild 3 -14: _dbi_btconfig %{?_rpmdb_rebuild:%{__dbi_btconfig_rebuild}} %{!?_rpmdb_rebuild:%{__dbi_btconfig_current}} %{nil} -14: _dbi_config %{_dbi_htconfig} -14: _dbi_config_Depends %{_dbi_htconfig} temporary private -14: _dbi_config_Dirnames %{_dbi_btconfig} -14: _dbi_config_Installtid %{_dbi_btconfig} -14: _dbi_config_Packages %{_dbi_htconfig} lockdbfd -14: _dbi_config_Provideversion %{_dbi_btconfig} -14: _dbi_config_Removetid %{_dbi_btconfig} -14: _dbi_config_Requireversion %{_dbi_btconfig} -14: _dbi_htconfig %{?_rpmdb_rebuild:%{__dbi_htconfig_rebuild}} %{!?_rpmdb_rebuild:%{__dbi_htconfig_current}} %{nil} -14: _dbi_tags Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Depends:Pubkeys -14: _dbpath %{_var}/lib/rpm -14: _dbpath_rebuild %{_dbpath} -14: _defaultdocdir %{_usr}/share/doc -14: _dependency_whiteout libtermcap>bash modutils>vixie-cron ypbind>yp-tools ghostscript-fonts>ghostscript %{?_dependency_whiteout_fc3} %{?_dependency_whiteout_fc2} %{?_dependency_whiteout_fc1} %{?_dependency_whiteout_9} %{?_dependency_whiteout_8_0} %{?_dependency_whiteout_7_2} %{?_dependency_whiteout_7_1} %{?_dependency_whiteout_7_0} %{?_dependency_whiteout_6_2} %{?_dependency_whiteout_6_1} %{?_dependency_whiteout_5_2} %{nil} -14: _dependency_whiteout_7_0 pango-gtkbeta-devel>pango-gtkbeta XFree86>Mesa compat-glibc>db2 compat-glibc>db1 pam>initscripts initscripts>sysklogd -14: _dependency_whiteout_7_1 arts>kdelibs-sound -14: _dependency_whiteout_7_2 libgnomeprint15>gnome-print nautilus>nautilus-mozilla tcl>postgresql-tcl -14: _dependency_whiteout_8_0 mysql>perl-DBD-MySQL perl>perl-Filter perl>mrtg perl>mod_perl perl-Date-Calc>perl-Bit-Vector -14: _dependency_whiteout_fc3 coreutils>pam nautilus>nautilus-cd-burner aspell>aspell-en kernel>initscripts kernel-smp>initscripts xorg-x11-libs>xorg-x11-Mesa-libGL openldap>cyrus-sasl-md5 openldap>cyrus-sasl openjade>docbook-dtds gtk+>gdk-pixbuf xorg-x11>xinitrc gnome-python2>gnome-python2-bonobo httpd-suexec>httpd xemacs-sumo>apel-xemacs php>php-pear openoffice.org-libs>openoffice.org -14: _desktopdir %{_datadir}/applications -14: _enable_debug_packages 1 -14: _exec_prefix %{_prefix} -14: _filter_GLIBC_PRIVATE 0 -14: _fixgroup [ `%{__id_u}` = '0' ] && %{__chgrp_Rhf} root -14: _fixowner [ `%{__id_u}` = '0' ] && %{__chown_Rhf} root -14: _fixperms %{__chmod} -Rf a+rX,u+w,g-w,o-w -14: _gcj_support 1 -14: _gnu -gnu -14: _gzipbin %{__gzip} -14: _hkp_keyserver %{nil} -14: _host x86_64-redhat-linux-gnu -14: _host_alias %{nil} -14: _host_cpu x86_64 -14: _host_os linux-gnu -14: _host_vendor redhat -14: _i18ndomains redhat-dist -14: _icons16dir %{_iconsbasedir}/16x16/apps -14: _icons192dir %{_iconsbasedir}/192x192/apps -14: _icons22dir %{_iconsbasedir}/22x22/apps -14: _icons48dir %{_iconsbasedir}/48x48/apps -14: _icons64dir %{_iconsbasedir}/64x64/apps -14: _icons96dir %{_iconsbasedir}/96x96/apps -14: _iconsbasedir %{_datadir}/icons/%{_iconstheme} -14: _iconsdir %{_datadir}/icons -14: _iconsscaldir %{_iconsbasedir}/scalable/apps -14: _iconstheme hicolor -14: _includedir %{_prefix}/include -14: _infodir /usr/share/info -14: _initrddir %{_sysconfdir}/rc.d/init.d -14: _install_file_context_path %{__file_context_path} -14: _install_langs all -14: _install_script_path /sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin -14: _instchangelog 5 -14: _javadir %{_datadir}/java -14: _javadocdir %{_datadir}/javadoc -14: _jnidir %{_prefix}/lib/java -14: _jvmcommondatadir %{_datadir}/jvm-commmon -14: _jvmcommonlibdir %{_prefix}/lib/jvm-commmon -14: _jvmcommonsysconfdir %{_sysconfdir}/jvm-commmon -14: _jvmdatadir %{_datadir}/jvm -14: _jvmdir %{_prefix}/lib/jvm -14: _jvmjardir %{_prefix}/lib/jvm-exports -14: _jvmlibdir %{_prefix}/lib/jvm -14: _jvmprivdir %{_prefix}/lib/jvm-private -14: _jvmsysconfdir %{_sysconfdir}/jvm -14: _lib lib64 -14: _libdir %{_exec_prefix}/%{_lib} -14: _libexecdir %{_exec_prefix}/libexec -14: _liconsdir %{_datadir}/icons/large -14: _localstatedir /var -14: _mandir /usr/share/man -14: _mavendepmapdir /etc/maven -14: _mavendepmapfragdir /etc/maven/fragments -14: _menudir %{_prefix}/lib/menu -14: _miconsdir %{_datadir}/icons/mini -14: _missing_doc_files_terminate_build 0 -14: _multilibno 2 -14: _multilibpatt (/%{_lib}|/usr/%{_lib}(|/gconv)|/usr/local/%{_lib}|/usr/X11R6/%{_lib}|/opt/%{_lib})/[^/]*\.([oa]|la|so[0-9.]*)$ -14: _oldincludedir /usr/include -14: _os linux -14: _package_version 30005 -14: _pgpbin %{__pgp} -15: _preScriptEnvironment RPM_SOURCE_DIR="%{_sourcedir}" RPM_BUILD_DIR="%{_builddir}" RPM_OPT_FLAGS="%{optflags}" RPM_ARCH="%{_arch}" RPM_OS="%{_os}" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{name}" RPM_PACKAGE_VERSION="%{version}" RPM_PACKAGE_RELEASE="%{release}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE %{?buildroot:RPM_BUILD_ROOT="%{buildroot}" export RPM_BUILD_ROOT } -14: _prefix /usr -14: _query_all_fmt %%{name}-%%{version}-%%{release} -14: _query_selector_match default -14: _repackage_all_erasures 0 -14: _repackage_dir /var/spool/repackage -14: _repackage_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm -14: _repackage_root %{nil} -14: _rollback_transaction_on_failure 0 -14: _rpmdir %{_topdir}/RPMS -14: _rpmfilename %{_build_name_fmt} -14: _rpmlock_path %{_dbpath}/__db.000 -14: _sbindir %{_exec_prefix}/sbin -14: _sharedstatedir %{_prefix}/com -14: _signature none -14: _smp_mflags %([ -z "$RPM_BUILD_NCPUS" ] \ && RPM_BUILD_NCPUS="`/usr/bin/getconf _NPROCESSORS_ONLN`"; \ [ "$RPM_BUILD_NCPUS" -gt 1 ] && echo "-j$RPM_BUILD_NCPUS") -14: _sourcedir %{_topdir}/SOURCES -14: _specdir %{_topdir}/SPECS -14: _srcrpmdir %{_topdir}/SRPMS -14: _sysconfdir /etc -11: _target x86_64-linux -14: _target_alias %{_host_alias} -11= _target_cpu x86_64 -11= _target_os linux -14: _target_platform %{_target_cpu}-%{_vendor}-%{_target_os}%{?_gnu} -14: _target_vendor %{_host_vendor} -14: _tmppath %{_var}/tmp -14: _topdir %{_usrsrc}/redhat -14: _transaction_color 3 -14: _unpackaged_files_terminate_build 0 -14: _unzipbin %{__unzip} -14: _use_internal_dependency_generator 1 -14: _usr /usr -14: _usrsrc %{_usr}/src -14: _var /var -14: _vendor redhat -14: _verify_file_context_path %{__file_context_path} -14: _vsflags_build %{__vsflags} -14: _vsflags_erase %{__vsflags} -14: _vsflags_install %{__vsflags} -14: _vsflags_query %{__vsflags} -14: _vsflags_rebuilddb %{__vsflags} -14: _vsflags_up2date %{__vsflags} -14: _vsflags_verify %{__vsflags} -14: _without_check 0 -14: add_jvm_extension JAVA_LIBDIR=%{buildroot}/%{_javadir} %{_bindir}/jvmjar -l -14: add_to_maven_depmap install -dm 755 $RPM_BUILD_ROOT/%{_mavendepmapfragdir} cat >>$RPM_BUILD_ROOT/%{_mavendepmapfragdir}/%{name}<< EOF %1 %2 %3 %4 %5 %3 EOF %{nil} -14: ant JAVA_HOME=%{java_home} ant -14: bcond_with %{expand:%%{?_with_%{1}:%%global with_%{1} 1}} -14: bcond_without %{expand:%%{!?_without_%{1}:%%global with_%{1} 1}} -14: build %%build LANG=C export LANG unset DISPLAY %{nil} -14: configure CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS ; CXXFLAGS="${CXXFLAGS:-%optflags}" ; export CXXFLAGS ; FFLAGS="${FFLAGS:-%optflags}" ; export FFLAGS ; for i in $(find . -name config.guess -o -name config.sub) ; do [ -f /usr/lib/rpm/redhat/$(basename $i) ] && %{__rm} -f $i && %{__cp} -fv /usr/lib/rpm/redhat/$(basename $i) $i ; done ; ./configure --build=%{_build} --host=%{_host} \ --target=%{_target_platform} \ --program-prefix=%{?_program_prefix} \ --prefix=%{_prefix} \ --exec-prefix=%{_exec_prefix} \ --bindir=%{_bindir} \ --sbindir=%{_sbindir} \ --sysconfdir=%{_sysconfdir} \ --datadir=%{_datadir} \ --includedir=%{_includedir} \ --libdir=%{_libdir} \ --libexecdir=%{_libexecdir} \ --localstatedir=%{_localstatedir} \ --sharedstatedir=%{_sharedstatedir} \ --mandir=%{_mandir} \ --infodir=%{_infodir} -14: debug_package %{nil} -14: defined %{expand:%%{?%{1}:1}%%{!?%{1}:0}} -14: distribution Rocks -14: find_lang /usr/lib/rpm/redhat/find-lang.sh %{buildroot} -14: install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}} %%install LANG=C export LANG unset DISPLAY %{nil} -14: ix86 i386 i486 i586 i686 pentium3 pentium4 athlon -14: jar %{java_home}/bin/jar -14: java %(. %{_javadir}-utils/java-functions; set_javacmd; echo $JAVACMD) -14: java_home %(. %{_javadir}-utils/java-functions; set_jvm; echo $JAVA_HOME) -14: javac %{java_home}/bin/javac -14: javadoc %{java_home}/bin/javadoc -14: jpackage_script install -d -m 755 $RPM_BUILD_ROOT%{_bindir} cat > $RPM_BUILD_ROOT%{_bindir}/%5 << EOF #!/bin/sh # # %{name} script # JPackage Project # Source functions library . %{_javadir}-utils/java-functions # Source system prefs if [ -f %{_sysconfdir}/java/%{name}.conf ] ; then . %{_sysconfdir}/java/%{name}.conf fi # Source user prefs if [ -f \$HOME/.%{name}rc ] ; then . \$HOME/.%{name}rc fi # Configuration MAIN_CLASS=%1 BASE_FLAGS=%2 BASE_OPTIONS=%3 BASE_JARS="%(echo %4 | sed -e 's,:, ,g')" # Set parameters set_jvm set_classpath \$BASE_JARS set_flags \$BASE_FLAGS set_options \$BASE_OPTIONS # Let's start run "\$@" EOF -14: kernel_module_package(n:v:r:s:f:xp:) %{expand:%( %define kmodtool %{-s*}%{!-s:/usr/lib/rpm/redhat/kmodtool} %define kmp_version %{-v*}%{!-v:%{version}} %define kmp_release %{-r*}%{!-r:%{release}} %define latest_kernel %(rpm -q --qf '%{VERSION}-%{RELEASE}\\n' `rpm -q kernel-devel | /usr/lib/rpm/redhat/rpmsort -r | head -n 1` | head -n 1) %{!?kernel_version:%{expand:%%global kernel_version %{latest_kernel}}} %global kverrel %(%{kmodtool} verrel %{?kernel_version} 2>/dev/null) flavors="default" if [ "i686" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) kdver=$(rpm -q kernel-kdump-devel-%{kverrel}|head -n 1) paever=$(rpm -q kernel-PAE-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi if [ "kernel-kdump-devel-%{kverrel}" == "$kdver" ] then flavors="$flavors kdump" fi if [ "kernel-PAE-devel-%{kverrel}" == "$paever" ] then flavors="$flavors PAE" fi fi if [ "x86_64" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi fi if [ "ppc64" == "%{_target_cpu}" ] || [ "ppc64iseries" == "%{_target_cpu}" ] then kdver=$(rpm -q kernel-kdump-devel-%{kverrel}|head -n 1) if [ "kernel-kdump-devel-%{kverrel}" == "$kdver" ] then flavors="$flavors kdump" fi fi if [ "ia64" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi fi if [ -z "%*" ]; then flavors_to_build=$flavors elif [ -z "%{-x}" ]; then flavors_to_build="%*" else flavors_to_build=" $flavors " echo "[$flavors_to_build]" >/tmp/tmp.txt for i in %* do flavors_to_build=${flavors_to_build//$i /} done fi echo "%%global flavors_to_build ${flavors_to_build:-%%nil}" echo "%%global kernel_source() /usr/src/kernels/%kverrel-\$([ %%%%{1} = default ] || echo "%%%%{1}-")%_target_cpu" if [ ! -z "%{-f*}" ] then filelist="%{-f*}" fi if [ ! -z "%{-p*}" ] then preamble="%{-p*}" fi if [ -z "%{kmodtool_generate_buildreqs}" ] then nobuildreqs="yes" fi kmp_override_filelist="$filelist" kmp_override_preamble="$preamble" kmp_nobuildreqs="$buildreqs" %{kmodtool} rpmtemplate_kmp %{-n*}%{!-n:%name} %{kverrel} $flavors_to_build 2>/dev/null )} -14: kernel_module_package_buildreqs %global kmodtool_generate_buildreqs 1 kernel-devel -14: makeinstall %{__make} \ prefix=%{?buildroot:%{buildroot}}%{_prefix} \ exec_prefix=%{?buildroot:%{buildroot}}%{_exec_prefix} \ bindir=%{?buildroot:%{buildroot}}%{_bindir} \ sbindir=%{?buildroot:%{buildroot}}%{_sbindir} \ sysconfdir=%{?buildroot:%{buildroot}}%{_sysconfdir} \ datadir=%{?buildroot:%{buildroot}}%{_datadir} \ includedir=%{?buildroot:%{buildroot}}%{_includedir} \ libdir=%{?buildroot:%{buildroot}}%{_libdir} \ libexecdir=%{?buildroot:%{buildroot}}%{_libexecdir} \ localstatedir=%{?buildroot:%{buildroot}}%{_localstatedir} \ sharedstatedir=%{?buildroot:%{buildroot}}%{_sharedstatedir} \ mandir=%{?buildroot:%{buildroot}}%{_mandir} \ infodir=%{?buildroot:%{buildroot}}%{_infodir} \ install -14: mdkicons install -D -m 644 %1-48.png $RPM_BUILD_ROOT%{_liconsdir}/%1.png install -D -m 644 %1-32.png $RPM_BUILD_ROOT%{_iconsdir}/%1.png install -D -m 644 %1-16.png $RPM_BUILD_ROOT%{_miconsdir}/%1.png -14: mdkmenu install -d -m 755 $RPM_BUILD_ROOT%{_menudir} cat >$RPM_BUILD_ROOT%{_menudir}/%1 <$RPM_BUILD_ROOT%{_sysconfdir}/X11/applnk/%4/%1.desktop <\n" > %{_mavendepmapdir}/maven2-depmap.xml if [ -d %{_mavendepmapfragdir} ] && [ -n "`find %{_mavendepmapfragdir} -type f`" ]; then cat %{_mavendepmapfragdir}/* >> %{_mavendepmapdir}/maven2-depmap.xml fi echo -e "\n" >> %{_mavendepmapdir}/maven2-depmap.xml -14: with %{expand:%%{?with_%{1}:1}%%{!?with_%{1}:0}} -14: without %{expand:%%{?with_%{1}:0}%%{!?with_%{1}:1}} ======================== active 327 empty 0 Thank you, Amit ________________________________ From: Kumar, Amit H. Sent: Monday, July 28, 2008 11:56 AM To: Vladimir Sokolovsky Cc: general at lists.openfabrics.org Subject: RE: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created # rpm --showrc ARCHITECTURE AND OS: build arch : x86_64 compatible build archs: ia32e x86_64 noarch build os : Linux compatible build os's : linux install arch : ia32e install os : Linux compatible archs : ia32e x86_64 athlon noarch amd64 i686 i586 i486 i386 fat compatible os's : linux RPMRC VALUES: macrofiles : /usr/lib/rpm/macros:/usr/lib/rpm/x86_64-linux/macros:/usr/lib/rpm/redhat/macros:/etc/rpm/macros.*:/etc/rpm/macros:/etc/rpm/x86_64-linux/macros:~/.rpmmacros optflags : -O2 -g Features supported by rpmlib: rpmlib(VersionedDependencies) = 3.0.3-1 PreReq:, Provides:, and Obsoletes: dependencies support versions. rpmlib(CompressedFileNames) = 3.0.4-1 file name(s) stored as (dirName,baseName,dirIndex) tuple, not as path. rpmlib(PayloadIsBzip2) = 3.0.5-1 package payload can be compressed using bzip2. rpmlib(PayloadFilesHavePrefix) = 4.0-1 package payload file(s) have "./" prefix. rpmlib(ExplicitPackageProvide) = 4.0-1 package name-version-release is not implicitly provided. rpmlib(HeaderLoadSortsTags) = 4.0.1-1 header tags are always sorted after being loaded. rpmlib(ScriptletInterpreterArgs) = 4.0.3-1 the scriptlet interpreter can use arguments from header. rpmlib(PartialHardlinkSets) = 4.0.4-1 a hardlink file set may be installed without being complete. rpmlib(ConcurrentAccess) = 4.1-1 package scriptlets may access the rpm database while installing. rpmlib(BuiltinLuaScripts) = 4.2.2-1 internal support for lua scripts. ======================== -14: GNUconfigure(MCs:) CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS; LDFLAGS="${LDFLAGS:-%{-s:-s}}" ; export LDFLAGS; %{-C:_mydir="`pwd`"; %{-M: %{__mkdir} -p %{-C*};} cd %{-C*}} dirs="`find ${_mydir} -name configure.in -print`"; export dirs; for coin in `echo ${dirs}` do dr=`dirname ${coin}`; if test -f ${dr}/NO-AUTO-GEN; then : else macrodirs=`sed -n -e 's,AM_ACLOCAL_INCLUDE((.*)),1,gp' < ${coin}`; ( cd ${dr}; aclocalinclude="${ACLOCAL_FLAGS}"; for k in ${macrodirs}; do if test -d ${k}; then aclocalinclude="${aclocalinclude} -I ${k}"; ##else ## echo "**Warning**: No such directory `${k}'. Ignored." fi done if grep "^AM_GNU_GETTEXT" configure.in >/dev/null; then if grep "sed.*POTFILES" configure.in >/dev/null; then : do nothing -- we still have an old unmodified configure.in else test -r ${dr}/aclocal.m4 || touch ${dr}/aclocal.m4; echo "no" | gettextize --force --copy; test -r ${dr}/aclocal.m4 && %{__chmod} u+w ${dr}/aclocal.m4; fi fi if grep "^AM_PROG_LIBTOOL" configure.in >/dev/null; then %{__libtoolize} --force --copy; fi aclocal ${aclocalinclude}; if grep "^AM_CONFIG_HEADER" configure.in >/dev/null; then %{__autoheader}; fi echo "Running automake --gnu ${am_opt} ..."; %{__automake} --add-missing --gnu ${am_opt}; %{__autoconf}; ); fi done %{-C:${_mydir}}%{!-C:.}/configure %{_target_platform} --prefix=%{_prefix} --exec-prefix=%{_exec_prefix} --bindir=%{_bindir} --sbindir=%{_sbindir} --sysconfdir=%{_sysconfdir} --datadir=%{_datadir} --includedir=%{_includedir} --libdir=%{_libdir} --libexecdir=%{_libexecdir} --localstatedir=%{_localstatedir} --sharedstatedir=%{_sharedstatedir} --mandir=%{_mandir} --infodir=%{_infodir} %* ; %{-C:cd ${_mydir}; unset _mydir} -14: ___build_args -e -14: ___build_cmd %{?_sudo:%{_sudo} }%{?_remsh:%{_remsh} %{_remhost} }%{?_remsudo:%{_remsudo} }%{?_remchroot:%{_remchroot} %{_remroot} }%{___build_shell} %{___build_args} -14: ___build_post exit 0 -14: ___build_pre RPM_SOURCE_DIR="%{u2p:%{_sourcedir}}" RPM_BUILD_DIR="%{u2p:%{_builddir}}" RPM_OPT_FLAGS="%{optflags}" RPM_ARCH="%{_arch}" RPM_OS="%{_os}" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{name}" RPM_PACKAGE_VERSION="%{version}" RPM_PACKAGE_RELEASE="%{release}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE %{?buildroot:RPM_BUILD_ROOT="%{u2p:%{buildroot}}" export RPM_BUILD_ROOT} %{?_javaclasspath:CLASSPATH="%{_javaclasspath}" export CLASSPATH} %{verbose:set -x}%{!verbose:exec > /dev/null} umask 022 cd %{u2p:%{_builddir}} -14: ___build_shell %{?_buildshell:%{_buildshell}}%{!?_buildshell:/bin/sh} -14: ___build_template #!%{___build_shell} %{___build_pre} %{nil} -14: __aclocal aclocal -14: __ar ar -14: __arch_install_post %{nil} -14: __as as -14: __autoconf autoconf -14: __autoheader autoheader -14: __automake automake -14: __awk gawk -14: __bzip2 /usr/bin/bzip2 -14: __cat /bin/cat -14: __cc gcc -14: __check_files /usr/lib/rpm/check-files %{buildroot} -14: __chgrp /bin/chgrp -14: __chgrp_Rhf %{__chgrp} -Rhf -14: __chmod /bin/chmod -14: __chown /bin/chown -14: __chown_Rhf %{__chown} -Rhf -14: __cp /bin/cp -14: __cpio /bin/cpio -14: __cpp gcc -E -14: __cxx g++ -14: __dbi_btconfig btree %{__dbi_other} %{__dbi_perms} %{nil} -14: __dbi_btconfig_current %{__dbi_btconfig} -14: __dbi_btconfig_rebuild %{__dbi_btconfig} %{__dbi_rebuild} -14: __dbi_cdb create cdb mpool mp_mmapsize=16Mb mp_size=1Mb -14: __dbi_htconfig hash %{__dbi_other} %{__dbi_perms} %{nil} -14: __dbi_htconfig_current %{__dbi_htconfig} -14: __dbi_htconfig_rebuild %{__dbi_htconfig} %{__dbi_rebuild} -14: __dbi_other %{?_tmppath:tmpdir=%{_tmppath}} %{?__dbi_cdb} -14: __dbi_perms perms=0644 -14: __dbi_rebuild nofsync !log !txn !cdb -14: __dbi_transient %{__dbi_rebuild} temporary private -14: __debug_install_post /usr/lib/rpm/find-debuginfo.sh %{_builddir}/%{?buildsubdir} %{nil} -14: __file /usr/bin/file -14: __file_context_path /etc/selinux/%{__policy_tree}/contexts/files/file_contexts -14: __find_provides /usr/lib/rpm/redhat/find-provides -14: __find_requires /usr/lib/rpm/redhat/find-requires -14: __global_cflags -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -14: __gpg /usr/bin/gpg -14: __gpg_check_password_cmd %{__gpg} gpg --batch --no-verbose --passphrase-fd 3 -u "%{_gpg_name}" -so - -14: __gpg_sign_cmd %{__gpg} gpg --batch --no-verbose --no-armor --passphrase-fd 3 --no-secmem-warning -u "%{_gpg_name}" -sbo %{__signature_filename} %{__plaintext_filename} -14: __grep /bin/grep -14: __gzip /bin/gzip -14: __id /usr/bin/id -14: __id_u %{__id} -u -14: __install /usr/bin/install -14: __ld /usr/bin/ld -14: __libtoolize libtoolize -14: __ln_s ln -s -14: __make /usr/bin/make -14: __mkdir /bin/mkdir -14: __mkdir_p /bin/mkdir -p -14: __mono_provides /usr/lib/rpm/mono-find-provides %{_builddir}/%{?buildsubdir} %{buildroot} %{_libdir} -14: __mono_requires /usr/lib/rpm/mono-find-requires %{_builddir}/%{?buildsubdir} %{buildroot} %{_libdir} -14: __mv /bin/mv -14: __nm /usr/bin/nm -14: __objcopy /usr/bin/objcopy -14: __objdump /usr/bin/objdump -14: __os_install_post /usr/lib/rpm/redhat/brp-compress %{!?__debug_package:/usr/lib/rpm/redhat/brp-strip %{__strip}} /usr/lib/rpm/redhat/brp-strip-static-archive %{__strip} /usr/lib/rpm/redhat/brp-strip-comment-note %{__strip} %{__objdump} /usr/lib/rpm/brp-python-bytecompile /usr/lib/rpm/redhat/brp-java-repack-jars %{nil} -14: __patch /usr/bin/patch -14: __perl /usr/bin/perl -14: __perl_provides /usr/lib/rpm/perl.prov -14: __perl_requires /usr/lib/rpm/perl.req -14: __pgp /usr/bin/pgp -14: __pgp5_check_password_cmd %{__pgp} pgps +batchmode=on +verbose=0 +armor=off "%{_pgp_name}" -f -14: __pgp5_sign_cmd %{__pgp} pgps +batchmode=on +verbose=0 +armor=off "+myname=%{_pgp_name}" -b %{__plaintext_filename} -o %{__signature_filename} -14: __pgp_check_password_cmd %{__pgp} pgp +batchmode=on +verbose=0 "%{_pgp_name}" -sf -14: __pgp_sign_cmd %{__pgp} pgp +batchmode=on +verbose=0 +armor=off "+myname=%{_pgp_name}" -sb %{__plaintext_filename} %{__signature_filename} -14: __policy_tree %{expand:%%global __policy_tree %{lua: t="targeted" f = io.open("/etc/selinux/config") if f then for l in f:lines() do if "SELINUXTYPE=" == string.sub(l,0,12) then t=string.sub(l,13); end end f:close() end print (t) }}%{__policy_tree} -14: __prelink_undo_cmd /usr/sbin/prelink prelink -y library -14: __python /usr/bin/python -14: __python_provides /usr/lib/rpm/pythondeps.sh --provides -14: __python_requires /usr/lib/rpm/pythondeps.sh --requires -14: __ranlib ranlib -14: __remsh %{__rsh} -14: __rm /bin/rm -14: __rsh /usr/bin/rsh -14: __sed /bin/sed -14: __spec_build_args %{___build_args} -14: __spec_build_body %{___build_body} -14: __spec_build_cmd %{___build_cmd} -14: __spec_build_post %{___build_post} -14: __spec_build_pre %{___build_pre} -14: __spec_build_shell %{___build_shell} -14: __spec_build_template #!%{__spec_build_shell} %{__spec_build_pre} %{nil} -14: __spec_check_args %{___build_args} -14: __spec_check_body %{___build_body} -14: __spec_check_cmd %{___build_cmd} -14: __spec_check_post %{___build_post} -14: __spec_check_pre %{___build_pre} -14: __spec_check_shell %{___build_shell} -14: __spec_check_template #!%{__spec_check_shell} %{__spec_check_pre} %{nil} -14: __spec_clean_args %{___build_args} -14: __spec_clean_body %{___build_body} -14: __spec_clean_cmd %{___build_cmd} -14: __spec_clean_post %{___build_post} -14: __spec_clean_pre %{___build_pre} -14: __spec_clean_shell %{___build_shell} -14: __spec_clean_template #!%{__spec_clean_shell} %{__spec_clean_pre} %{nil} -14: __spec_install_args %{___build_args} -14: __spec_install_body %{___build_body} -14: __spec_install_cmd %{___build_cmd} -14: __spec_install_post %{?__debug_package:%{__debug_install_post}} %{__arch_install_post} %{__os_install_post} %{nil} -14: __spec_install_pre %{___build_pre} -14: __spec_install_shell %{___build_shell} -14: __spec_install_template #!%{__spec_install_shell} %{__spec_install_pre} %{nil} -14: __spec_prep_args %{___build_args} -14: __spec_prep_body %{___build_body} -14: __spec_prep_cmd %{___build_cmd} -14: __spec_prep_post %{___build_post} -14: __spec_prep_pre %{___build_pre} -14: __spec_prep_shell %{___build_shell} -14: __spec_prep_template #!%{__spec_prep_shell} %{__spec_prep_pre} %{nil} -14: __spec_rmbuild_args %{___build_args} -14: __spec_rmbuild_body %{___build_body} -14: __spec_rmbuild_cmd %{___build_cmd} -14: __spec_rmbuild_post %{___build_post} -14: __spec_rmbuild_pre %{___build_pre} -14: __spec_rmbuild_shell %{___build_shell} -14: __spec_rmbuild_template #!%{__spec_rmbuild_shell} %{__spec_rmbuild_pre} %{nil} -14: __ssh /usr/bin/ssh -14: __strip /usr/bin/strip -14: __tar /bin/tar -14: __unzip /usr/bin/unzip -14: __vsflags 0xc0c00 -14: _arch x86_64 -14: _autorelocate_dcolor 0 -14: _autorelocate_path %{nil} -14: _bhA RPMS -14: _bhN @(SRPMS|i386|alpha|sparc|s390|ia64) -14: _bhVR RedHat -14: _bhcoll @(7.3|7.2|7.1|7.1sbe|7.1k|7.0|7.01j|7.0j|7.0sbe|7.0tc|6.2|6.2ha|6.2ee|6.1|6.0|5.2|5.1|5.0) -14: _bhpath file://localhost/mnt/dist -14: _bindir %{_exec_prefix}/bin -14: _build %{_host} -14: _build_alias %{_host_alias} -14: _build_arch x86_64 -14: _build_cpu %{_host_cpu} -14: _build_file_context_path %{nil} -14: _build_name_fmt %%{ARCH}/%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm -14: _build_os %{_host_os} -14: _build_vendor %{_host_vendor} -14: _builddir %{_topdir}/BUILD -14: _buildshell /bin/sh -14: _bzip2bin %{__bzip2} -14: _cache_dbpath /var/spool/up2date/cache -14: _datadir %{_prefix}/share -14: _dbapi 3 -14: _dbapi_rebuild 3 -14: _dbi_btconfig %{?_rpmdb_rebuild:%{__dbi_btconfig_rebuild}} %{!?_rpmdb_rebuild:%{__dbi_btconfig_current}} %{nil} -14: _dbi_config %{_dbi_htconfig} -14: _dbi_config_Depends %{_dbi_htconfig} temporary private -14: _dbi_config_Dirnames %{_dbi_btconfig} -14: _dbi_config_Installtid %{_dbi_btconfig} -14: _dbi_config_Packages %{_dbi_htconfig} lockdbfd -14: _dbi_config_Provideversion %{_dbi_btconfig} -14: _dbi_config_Removetid %{_dbi_btconfig} -14: _dbi_config_Requireversion %{_dbi_btconfig} -14: _dbi_htconfig %{?_rpmdb_rebuild:%{__dbi_htconfig_rebuild}} %{!?_rpmdb_rebuild:%{__dbi_htconfig_current}} %{nil} -14: _dbi_tags Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Depends:Pubkeys -14: _dbpath %{_var}/lib/rpm -14: _dbpath_rebuild %{_dbpath} -14: _defaultdocdir %{_usr}/share/doc -14: _dependency_whiteout libtermcap>bash modutils>vixie-cron ypbind>yp-tools ghostscript-fonts>ghostscript %{?_dependency_whiteout_fc3} %{?_dependency_whiteout_fc2} %{?_dependency_whiteout_fc1} %{?_dependency_whiteout_9} %{?_dependency_whiteout_8_0} %{?_dependency_whiteout_7_2} %{?_dependency_whiteout_7_1} %{?_dependency_whiteout_7_0} %{?_dependency_whiteout_6_2} %{?_dependency_whiteout_6_1} %{?_dependency_whiteout_5_2} %{nil} -14: _dependency_whiteout_7_0 pango-gtkbeta-devel>pango-gtkbeta XFree86>Mesa compat-glibc>db2 compat-glibc>db1 pam>initscripts initscripts>sysklogd -14: _dependency_whiteout_7_1 arts>kdelibs-sound -14: _dependency_whiteout_7_2 libgnomeprint15>gnome-print nautilus>nautilus-mozilla tcl>postgresql-tcl -14: _dependency_whiteout_8_0 mysql>perl-DBD-MySQL perl>perl-Filter perl>mrtg perl>mod_perl perl-Date-Calc>perl-Bit-Vector -14: _dependency_whiteout_fc3 coreutils>pam nautilus>nautilus-cd-burner aspell>aspell-en kernel>initscripts kernel-smp>initscripts xorg-x11-libs>xorg-x11-Mesa-libGL openldap>cyrus-sasl-md5 openldap>cyrus-sasl openjade>docbook-dtds gtk+>gdk-pixbuf xorg-x11>xinitrc gnome-python2>gnome-python2-bonobo httpd-suexec>httpd xemacs-sumo>apel-xemacs php>php-pear openoffice.org-libs>openoffice.org -14: _desktopdir %{_datadir}/applications -14: _enable_debug_packages 1 -14: _exec_prefix %{_prefix} -14: _filter_GLIBC_PRIVATE 0 -14: _fixgroup [ `%{__id_u}` = '0' ] && %{__chgrp_Rhf} root -14: _fixowner [ `%{__id_u}` = '0' ] && %{__chown_Rhf} root -14: _fixperms %{__chmod} -Rf a+rX,u+w,g-w,o-w -14: _gcj_support 1 -14: _gnu -gnu -14: _gzipbin %{__gzip} -14: _hkp_keyserver %{nil} -14: _host x86_64-redhat-linux-gnu -14: _host_alias %{nil} -14: _host_cpu x86_64 -14: _host_os linux-gnu -14: _host_vendor redhat -14: _i18ndomains redhat-dist -14: _icons16dir %{_iconsbasedir}/16x16/apps -14: _icons192dir %{_iconsbasedir}/192x192/apps -14: _icons22dir %{_iconsbasedir}/22x22/apps -14: _icons48dir %{_iconsbasedir}/48x48/apps -14: _icons64dir %{_iconsbasedir}/64x64/apps -14: _icons96dir %{_iconsbasedir}/96x96/apps -14: _iconsbasedir %{_datadir}/icons/%{_iconstheme} -14: _iconsdir %{_datadir}/icons -14: _iconsscaldir %{_iconsbasedir}/scalable/apps -14: _iconstheme hicolor -14: _includedir %{_prefix}/include -14: _infodir /usr/share/info -14: _initrddir %{_sysconfdir}/rc.d/init.d -14: _install_file_context_path %{__file_context_path} -14: _install_langs all -14: _install_script_path /sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin -14: _instchangelog 5 -14: _javadir %{_datadir}/java -14: _javadocdir %{_datadir}/javadoc -14: _jnidir %{_prefix}/lib/java -14: _jvmcommondatadir %{_datadir}/jvm-commmon -14: _jvmcommonlibdir %{_prefix}/lib/jvm-commmon -14: _jvmcommonsysconfdir %{_sysconfdir}/jvm-commmon -14: _jvmdatadir %{_datadir}/jvm -14: _jvmdir %{_prefix}/lib/jvm -14: _jvmjardir %{_prefix}/lib/jvm-exports -14: _jvmlibdir %{_prefix}/lib/jvm -14: _jvmprivdir %{_prefix}/lib/jvm-private -14: _jvmsysconfdir %{_sysconfdir}/jvm -14: _lib lib64 -14: _libdir %{_exec_prefix}/%{_lib} -14: _libexecdir %{_exec_prefix}/libexec -14: _liconsdir %{_datadir}/icons/large -14: _localstatedir /var -14: _mandir /usr/share/man -14: _mavendepmapdir /etc/maven -14: _mavendepmapfragdir /etc/maven/fragments -14: _menudir %{_prefix}/lib/menu -14: _miconsdir %{_datadir}/icons/mini -14: _missing_doc_files_terminate_build 0 -14: _multilibno 2 -14: _multilibpatt (/%{_lib}|/usr/%{_lib}(|/gconv)|/usr/local/%{_lib}|/usr/X11R6/%{_lib}|/opt/%{_lib})/[^/]*\.([oa]|la|so[0-9.]*)$ -14: _oldincludedir /usr/include -14: _os linux -14: _package_version 30005 -14: _pgpbin %{__pgp} -15: _preScriptEnvironment RPM_SOURCE_DIR="%{_sourcedir}" RPM_BUILD_DIR="%{_builddir}" RPM_OPT_FLAGS="%{optflags}" RPM_ARCH="%{_arch}" RPM_OS="%{_os}" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{name}" RPM_PACKAGE_VERSION="%{version}" RPM_PACKAGE_RELEASE="%{release}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE %{?buildroot:RPM_BUILD_ROOT="%{buildroot}" export RPM_BUILD_ROOT } -14: _prefix /usr -14: _query_all_fmt %%{name}-%%{version}-%%{release} -14: _query_selector_match default -14: _repackage_all_erasures 0 -14: _repackage_dir /var/spool/repackage -14: _repackage_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm -14: _repackage_root %{nil} -14: _rollback_transaction_on_failure 0 -14: _rpmdir %{_topdir}/RPMS -14: _rpmfilename %{_build_name_fmt} -14: _rpmlock_path %{_dbpath}/__db.000 -14: _sbindir %{_exec_prefix}/sbin -14: _sharedstatedir %{_prefix}/com -14: _signature none -14: _smp_mflags %([ -z "$RPM_BUILD_NCPUS" ] \ && RPM_BUILD_NCPUS="`/usr/bin/getconf _NPROCESSORS_ONLN`"; \ [ "$RPM_BUILD_NCPUS" -gt 1 ] && echo "-j$RPM_BUILD_NCPUS") -14: _sourcedir %{_topdir}/SOURCES -14: _specdir %{_topdir}/SPECS -14: _srcrpmdir %{_topdir}/SRPMS -14: _sysconfdir /etc -11: _target x86_64-linux -14: _target_alias %{_host_alias} -11= _target_cpu x86_64 -11= _target_os linux -14: _target_platform %{_target_cpu}-%{_vendor}-%{_target_os}%{?_gnu} -14: _target_vendor %{_host_vendor} -14: _tmppath %{_var}/tmp -14: _topdir %{_usrsrc}/redhat -14: _transaction_color 3 -14: _unpackaged_files_terminate_build 0 -14: _unzipbin %{__unzip} -14: _use_internal_dependency_generator 1 -14: _usr /usr -14: _usrsrc %{_usr}/src -14: _var /var -14: _vendor redhat -14: _verify_file_context_path %{__file_context_path} -14: _vsflags_build %{__vsflags} -14: _vsflags_erase %{__vsflags} -14: _vsflags_install %{__vsflags} -14: _vsflags_query %{__vsflags} -14: _vsflags_rebuilddb %{__vsflags} -14: _vsflags_up2date %{__vsflags} -14: _vsflags_verify %{__vsflags} -14: _without_check 0 -14: add_jvm_extension JAVA_LIBDIR=%{buildroot}/%{_javadir} %{_bindir}/jvmjar -l -14: add_to_maven_depmap install -dm 755 $RPM_BUILD_ROOT/%{_mavendepmapfragdir} cat >>$RPM_BUILD_ROOT/%{_mavendepmapfragdir}/%{name}<< EOF %1 %2 %3 %4 %5 %3 EOF %{nil} -14: ant JAVA_HOME=%{java_home} ant -14: bcond_with %{expand:%%{?_with_%{1}:%%global with_%{1} 1}} -14: bcond_without %{expand:%%{!?_without_%{1}:%%global with_%{1} 1}} -14: build %%build LANG=C export LANG unset DISPLAY %{nil} -14: configure CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS ; CXXFLAGS="${CXXFLAGS:-%optflags}" ; export CXXFLAGS ; FFLAGS="${FFLAGS:-%optflags}" ; export FFLAGS ; for i in $(find . -name config.guess -o -name config.sub) ; do [ -f /usr/lib/rpm/redhat/$(basename $i) ] && %{__rm} -f $i && %{__cp} -fv /usr/lib/rpm/redhat/$(basename $i) $i ; done ; ./configure --build=%{_build} --host=%{_host} \ --target=%{_target_platform} \ --program-prefix=%{?_program_prefix} \ --prefix=%{_prefix} \ --exec-prefix=%{_exec_prefix} \ --bindir=%{_bindir} \ --sbindir=%{_sbindir} \ --sysconfdir=%{_sysconfdir} \ --datadir=%{_datadir} \ --includedir=%{_includedir} \ --libdir=%{_libdir} \ --libexecdir=%{_libexecdir} \ --localstatedir=%{_localstatedir} \ --sharedstatedir=%{_sharedstatedir} \ --mandir=%{_mandir} \ --infodir=%{_infodir} -14: debug_package %{nil} -14: defined %{expand:%%{?%{1}:1}%%{!?%{1}:0}} -14: distribution Rocks -14: find_lang /usr/lib/rpm/redhat/find-lang.sh %{buildroot} -14: install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}} %%install LANG=C export LANG unset DISPLAY %{nil} -14: ix86 i386 i486 i586 i686 pentium3 pentium4 athlon -14: jar %{java_home}/bin/jar -14: java %(. %{_javadir}-utils/java-functions; set_javacmd; echo $JAVACMD) -14: java_home %(. %{_javadir}-utils/java-functions; set_jvm; echo $JAVA_HOME) -14: javac %{java_home}/bin/javac -14: javadoc %{java_home}/bin/javadoc -14: jpackage_script install -d -m 755 $RPM_BUILD_ROOT%{_bindir} cat > $RPM_BUILD_ROOT%{_bindir}/%5 << EOF #!/bin/sh # # %{name} script # JPackage Project # Source functions library . %{_javadir}-utils/java-functions # Source system prefs if [ -f %{_sysconfdir}/java/%{name}.conf ] ; then . %{_sysconfdir}/java/%{name}.conf fi # Source user prefs if [ -f \$HOME/.%{name}rc ] ; then . \$HOME/.%{name}rc fi # Configuration MAIN_CLASS=%1 BASE_FLAGS=%2 BASE_OPTIONS=%3 BASE_JARS="%(echo %4 | sed -e 's,:, ,g')" # Set parameters set_jvm set_classpath \$BASE_JARS set_flags \$BASE_FLAGS set_options \$BASE_OPTIONS # Let's start run "\$@" EOF -14: kernel_module_package(n:v:r:s:f:xp:) %{expand:%( %define kmodtool %{-s*}%{!-s:/usr/lib/rpm/redhat/kmodtool} %define kmp_version %{-v*}%{!-v:%{version}} %define kmp_release %{-r*}%{!-r:%{release}} %define latest_kernel %(rpm -q --qf '%{VERSION}-%{RELEASE}\\n' `rpm -q kernel-devel | /usr/lib/rpm/redhat/rpmsort -r | head -n 1` | head -n 1) %{!?kernel_version:%{expand:%%global kernel_version %{latest_kernel}}} %global kverrel %(%{kmodtool} verrel %{?kernel_version} 2>/dev/null) flavors="default" if [ "i686" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) kdver=$(rpm -q kernel-kdump-devel-%{kverrel}|head -n 1) paever=$(rpm -q kernel-PAE-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi if [ "kernel-kdump-devel-%{kverrel}" == "$kdver" ] then flavors="$flavors kdump" fi if [ "kernel-PAE-devel-%{kverrel}" == "$paever" ] then flavors="$flavors PAE" fi fi if [ "x86_64" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi fi if [ "ppc64" == "%{_target_cpu}" ] || [ "ppc64iseries" == "%{_target_cpu}" ] then kdver=$(rpm -q kernel-kdump-devel-%{kverrel}|head -n 1) if [ "kernel-kdump-devel-%{kverrel}" == "$kdver" ] then flavors="$flavors kdump" fi fi if [ "ia64" == "%{_target_cpu}" ] then xenver=$(rpm -q kernel-xen-devel-%{kverrel}|head -n 1) if [ "kernel-xen-devel-%{kverrel}" == "$xenver" ] then flavors="$flavors xen" fi fi if [ -z "%*" ]; then flavors_to_build=$flavors elif [ -z "%{-x}" ]; then flavors_to_build="%*" else flavors_to_build=" $flavors " echo "[$flavors_to_build]" >/tmp/tmp.txt for i in %* do flavors_to_build=${flavors_to_build//$i /} done fi echo "%%global flavors_to_build ${flavors_to_build:-%%nil}" echo "%%global kernel_source() /usr/src/kernels/%kverrel-\$([ %%%%{1} = default ] || echo "%%%%{1}-")%_target_cpu" if [ ! -z "%{-f*}" ] then filelist="%{-f*}" fi if [ ! -z "%{-p*}" ] then preamble="%{-p*}" fi if [ -z "%{kmodtool_generate_buildreqs}" ] then nobuildreqs="yes" fi kmp_override_filelist="$filelist" kmp_override_preamble="$preamble" kmp_nobuildreqs="$buildreqs" %{kmodtool} rpmtemplate_kmp %{-n*}%{!-n:%name} %{kverrel} $flavors_to_build 2>/dev/null )} -14: kernel_module_package_buildreqs %global kmodtool_generate_buildreqs 1 kernel-devel -14: makeinstall %{__make} \ prefix=%{?buildroot:%{buildroot}}%{_prefix} \ exec_prefix=%{?buildroot:%{buildroot}}%{_exec_prefix} \ bindir=%{?buildroot:%{buildroot}}%{_bindir} \ sbindir=%{?buildroot:%{buildroot}}%{_sbindir} \ sysconfdir=%{?buildroot:%{buildroot}}%{_sysconfdir} \ datadir=%{?buildroot:%{buildroot}}%{_datadir} \ includedir=%{?buildroot:%{buildroot}}%{_includedir} \ libdir=%{?buildroot:%{buildroot}}%{_libdir} \ libexecdir=%{?buildroot:%{buildroot}}%{_libexecdir} \ localstatedir=%{?buildroot:%{buildroot}}%{_localstatedir} \ sharedstatedir=%{?buildroot:%{buildroot}}%{_sharedstatedir} \ mandir=%{?buildroot:%{buildroot}}%{_mandir} \ infodir=%{?buildroot:%{buildroot}}%{_infodir} \ install -14: mdkicons install -D -m 644 %1-48.png $RPM_BUILD_ROOT%{_liconsdir}/%1.png install -D -m 644 %1-32.png $RPM_BUILD_ROOT%{_iconsdir}/%1.png install -D -m 644 %1-16.png $RPM_BUILD_ROOT%{_miconsdir}/%1.png -14: mdkmenu install -d -m 755 $RPM_BUILD_ROOT%{_menudir} cat >$RPM_BUILD_ROOT%{_menudir}/%1 <$RPM_BUILD_ROOT%{_sysconfdir}/X11/applnk/%4/%1.desktop <\n" > %{_mavendepmapdir}/maven2-depmap.xml if [ -d %{_mavendepmapfragdir} ] && [ -n "`find %{_mavendepmapfragdir} -type f`" ]; then cat %{_mavendepmapfragdir}/* >> %{_mavendepmapdir}/maven2-depmap.xml fi echo -e "\n" >> %{_mavendepmapdir}/maven2-depmap.xml -14: with %{expand:%%{?with_%{1}:1}%%{!?with_%{1}:0}} -14: without %{expand:%%{?with_%{1}:0}%%{!?with_%{1}:1}} ======================== active 327 empty 0 Thank you, Amit ________________________________ From: general-bounces at lists.openfabrics.org [general-bounces at lists.openfabrics.org] On Behalf Of Vladimir Sokolovsky [vlad at mellanox.co.il] Sent: Monday, July 28, 2008 11:48 AM To: Kumar, Amit H. Cc: general at lists.openfabrics.org Subject: Re: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created Kumar, Amit H. wrote: > Hello ... > > Installing OFED 1.3.1 for the first time. I get the following message on > building RPMS and then installation stops. > > It creates Kernel-ib, kernel-ib-devel and ib-bonding rpms and then fails > at the following. > > Running rpmbuild --rebuild --define '_topdir /var/tmp/OFED_topdir' > --define 'KVERSION 2.6.18-53.1.21.el5' --define '_release > 2.6.18_53.1.21.el5' --define '_prefix /usr' > /share/apps/OFED-1.3.1/SRPMS/ib-bonding-0.9.0-25.src.rpm > ib-bonding-debuginfo was not created > > I then customised the installation to exclude ib-bonding-debuginfo, > installation continued and failed to build any debuginfo RPMS. Finally I > could complete the installation without any debuginfo libraries. > > Any idea what could I be doing wrong ??? > > Thank you, > Amit > Can you please send the output of 'rpm --showrc'? Regards, Vladimir _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Mon Jul 28 09:37:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 28 Jul 2008 19:37:05 +0300 Subject: ***SPAM*** Re: [ofa-general] ***SPAM*** [PATCH] opensm/*/Makefile.am: remove explicit -lpthread and -ldl flags from Makefile.am In-Reply-To: <20080728084821.48195028.weiny2@llnl.gov> References: <20080727130916.GT5067@sashak.voltaire.com> <20080727184705.GV5067@sashak.voltaire.com> <20080728084821.48195028.weiny2@llnl.gov> Message-ID: <20080728163705.GM14872@sashak.voltaire.com> Hi Ira, On 08:48 Mon 28 Jul , Ira Weiny wrote: > > diff --git a/opensm/configure.in b/opensm/configure.in > > index 7d04c14..7da932b 100644 > > --- a/opensm/configure.in > > +++ b/opensm/configure.in > > @@ -38,6 +38,8 @@ fi > > dnl Checks for libraries > > AC_CHECK_LIB(pthread, pthread_mutex_init, [], > > AC_MSG_ERROR([pthread_mutex_init() not found. libosmcomp requires libpthread.])) > > +AC_CHECK_LIB(dl, dlopen, [], > > + AC_MSG_ERROR([dlopen() not found. OpenSM requires libdl.])) > > > > dnl Checks for typedefs, structures, and compiler characteristics. > > AC_C_CONST > > I think it is still valid to check for the dl lib. Sure, so I add this check to configure.in - this should generate -ldl as part of LIBS variable. Sasha From weiny2 at llnl.gov Mon Jul 28 10:51:35 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 28 Jul 2008 10:51:35 -0700 Subject: ***SPAM*** Re: [ofa-general] ***SPAM*** [PATCH] opensm/*/Makefile.am: remove explicit -lpthread and -ldl flags from Makefile.am In-Reply-To: <20080728163705.GM14872@sashak.voltaire.com> References: <20080727130916.GT5067@sashak.voltaire.com> <20080727184705.GV5067@sashak.voltaire.com> <20080728084821.48195028.weiny2@llnl.gov> <20080728163705.GM14872@sashak.voltaire.com> Message-ID: <20080728105135.2fc65fc4.weiny2@llnl.gov> Oh, nevermind. I though you removed it... "-" vs "+"... Sorry, it was too early for my previous email... Ira On Mon, 28 Jul 2008 19:37:05 +0300 Sasha Khapyorsky wrote: > Hi Ira, > > On 08:48 Mon 28 Jul , Ira Weiny wrote: > > > diff --git a/opensm/configure.in b/opensm/configure.in > > > index 7d04c14..7da932b 100644 > > > --- a/opensm/configure.in > > > +++ b/opensm/configure.in > > > @@ -38,6 +38,8 @@ fi > > > dnl Checks for libraries > > > AC_CHECK_LIB(pthread, pthread_mutex_init, [], > > > AC_MSG_ERROR([pthread_mutex_init() not found. libosmcomp requires libpthread.])) > > > +AC_CHECK_LIB(dl, dlopen, [], > > > + AC_MSG_ERROR([dlopen() not found. OpenSM requires libdl.])) > > > > > > dnl Checks for typedefs, structures, and compiler characteristics. > > > AC_C_CONST > > > > I think it is still valid to check for the dl lib. > > Sure, so I add this check to configure.in - this should generate -ldl as > part of LIBS variable. > > Sasha From hal.rosenstock at gmail.com Mon Jul 28 12:56:36 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 28 Jul 2008 15:56:36 -0400 Subject: ***SPAM*** Re: [ofa-general] [PATCH V3] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM In-Reply-To: <20080728105316.16cd4bf9.weiny2@llnl.gov> References: <20080722172809.5d0831f3.weiny2@llnl.gov> <20080723080339.73f3914f.weiny2@llnl.gov> <20080724032517.GE11924@sashak.voltaire.com> <20080724090247.64d8c0e6.weiny2@llnl.gov> <20080724101113.76537e5e.weiny2@llnl.gov> <20080728105316.16cd4bf9.weiny2@llnl.gov> Message-ID: On Mon, Jul 28, 2008 at 1:53 PM, Ira Weiny wrote: > On Mon, 28 Jul 2008 10:02:15 -0400 > "Hal Rosenstock" wrote: > >> On Thu, Jul 24, 2008 at 1:11 PM, Ira Weiny wrote: >> > >From 1f810dd6ae9a73549cacdbb26f464e6132675592 Mon Sep 17 00:00:00 2001 >> > From: Ira K. Weiny >> > Date: Tue, 22 Jul 2008 16:31:36 -0700 >> > Subject: [PATCH] Add ib_trap_str function to ib_types.h and use it in INFO prints of OpenSM >> > >> > >> > Signed-off-by: Ira K. Weiny >> > --- >> > opensm/include/iba/ib_types.h | 2 + >> > opensm/opensm/libopensm.map | 1 + >> > opensm/opensm/osm_helper.c | 44 +++++++++++++++++++++++++++++++++++++++++ >> > opensm/opensm/osm_inform.c | 3 +- >> > opensm/opensm/osm_trap_rcv.c | 10 +++++++- >> > 5 files changed, 57 insertions(+), 3 deletions(-) >> > >> > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h >> > index 09ec257..fffce00 100644 >> > --- a/opensm/include/iba/ib_types.h >> > +++ b/opensm/include/iba/ib_types.h >> > @@ -7219,6 +7219,8 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated >> > #define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) >> > #define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) >> > >> > +char * OSM_API ib_get_trap_str(uint16_t trap_num); >> > + >> > /****f* IBA Base: Types/ib_notice_is_generic >> > * NAME >> > * ib_notice_is_generic >> > diff --git a/opensm/opensm/libopensm.map b/opensm/opensm/libopensm.map >> > index 759a4e8..7cd2aba 100644 >> > --- a/opensm/opensm/libopensm.map >> > +++ b/opensm/opensm/libopensm.map >> > @@ -17,6 +17,7 @@ OPENSM_1.5 { >> >> Doesn't this also need updating as an API is being added ? >> >> -- Hal > > Probably, do we have anything else we want to add/change? Per Sasha's mail, this doesn't need updating; only the API version (prior to tarball release). Who's going to remember to do this ? -- Hal > > Ira > >> >> > ib_get_sm_method_str; >> > ib_get_sm_attr_str; >> > ib_get_sa_attr_str; >> > + ib_get_trap_str; >> > osm_dump_port_info; >> > osm_dump_portinfo_record; >> > osm_dump_guidinfo_record; >> > diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c >> > index 21ff51c..cc76d5c 100644 >> > --- a/opensm/opensm/osm_helper.c >> > +++ b/opensm/opensm/osm_helper.c >> > @@ -2338,3 +2338,47 @@ const char *osm_get_sm_mgr_state_str(IN uint16_t state) >> > __osm_sm_mgr_state_str[state] : >> > __osm_sm_mgr_state_str[ARR_SIZE(__osm_sm_mgr_state_str) - 1]; >> > } >> > + >> > +char * OSM_API >> > +ib_get_trap_str(uint16_t trap_num) >> > +{ >> > + switch(trap_num) >> > + { >> > + case 64: >> > + return ("GID in service"); >> > + case 65: >> > + return ("GID out of service"); >> > + case 66: >> > + return ("New mcast group created"); >> > + case 67: >> > + return ("Mcast group deleted"); >> > + case 68: >> > + return ("UnPath, Path no longer valid"); >> > + case 69: >> > + return ("RePath, Path recomputed"); >> > + case 128: >> > + return ("Link state change"); >> > + case 129: >> > + return ("Local Link integrity threshold reached"); >> > + case 130: >> > + return ("Excessive Buffer Overrun Threshold reached"); >> > + case 131: >> > + return ("Flow Control Update watchdog timer expired"); >> > + case 144: >> > + return ("CapabilityMask, NodeDescription, Link [Width|Speed] Enabled changed"); >> > + case 145: >> > + return ("System Image GUID changed"); >> > + case 256: >> > + return ("Bad M_Key"); >> > + case 257: >> > + return ("Bad P_Key"); >> > + case 258: >> > + return ("Bad Q_Key"); >> > + case 259: >> > + return ("Bad P_Key (switch external port)"); >> > + default: >> > + break; >> > + } >> > + return ("Unknown"); >> > +} >> > + >> > diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c >> > index e6146f7..ebc13bb 100644 >> > --- a/opensm/opensm/osm_inform.c >> > +++ b/opensm/opensm/osm_inform.c >> > @@ -581,11 +581,12 @@ osm_report_notice(IN osm_log_t * const p_log, >> > /* an official Event information log */ >> > if (ib_notice_is_generic(p_ntc)) { >> > OSM_LOG(p_log, OSM_LOG_INFO, >> > - "Reporting Generic Notice type:%u num:%u" >> > + "Reporting Generic Notice type:%u num:%u (%s)" >> > " from LID:%u GID:0x%016" PRIx64 >> > ",0x%016" PRIx64 "\n", >> > ib_notice_get_type(p_ntc), >> > cl_ntoh16(p_ntc->g_or_v.generic.trap_num), >> > + ib_get_trap_str(cl_ntoh16(p_ntc->g_or_v.generic.trap_num)), >> > cl_ntoh16(p_ntc->issuer_lid), >> > cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), >> > cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) >> > diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c >> > index ae7ab05..03cbd26 100644 >> > --- a/opensm/opensm/osm_trap_rcv.c >> > +++ b/opensm/opensm/osm_trap_rcv.c >> > @@ -327,11 +327,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, >> > || (p_ntci->g_or_v.generic.trap_num == >> > CL_HTON16(131))) >> > OSM_LOG(sm->p_log, OSM_LOG_ERROR, >> > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " >> > + "Received Generic Notice type:0x%02X " >> > + "num:%u (%s) Producer:%u (%s) " >> > "from LID:%u Port %d TID:0x%016" >> > PRIx64 "\n", ib_notice_get_type(p_ntci), >> > cl_ntoh16(p_ntci->g_or_v.generic. >> > trap_num), >> > + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. >> > + trap_num)), >> > cl_ntoh32(ib_notice_get_prod_type >> > (p_ntci)), >> > ib_get_producer_type_str >> > @@ -342,11 +345,14 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, >> > ); >> > else >> > OSM_LOG(sm->p_log, OSM_LOG_ERROR, >> > - "Received Generic Notice type:0x%02X num:%u Producer:%u (%s) " >> > + "Received Generic Notice type:0x%02X " >> > + "num:%u (%s) Producer:%u (%s) " >> > "from LID:%u TID:0x%016" PRIx64 >> > "\n", ib_notice_get_type(p_ntci), >> > cl_ntoh16(p_ntci->g_or_v.generic. >> > trap_num), >> > + ib_get_trap_str(cl_ntoh16(p_ntci->g_or_v.generic. >> > + trap_num)), >> > cl_ntoh32(ib_notice_get_prod_type >> > (p_ntci)), >> > ib_get_producer_type_str >> > -- >> > 1.5.4.5 >> > >> > _______________________________________________ >> > general mailing list >> > general at lists.openfabrics.org >> > http:// lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> > >> > To unsubscribe, please visit http:// openib.org/mailman/listinfo/openib-general >> > >> > From vlad at mellanox.co.il Mon Jul 28 23:50:51 2008 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 29 Jul 2008 09:50:51 +0300 Subject: [ofa-general] OFED Installation Problems: ib-bonding-debuginfo was not created In-Reply-To: References: , <488DEA6B.1060409@mellanox.co.il>, <879DC79E-7C18-4C4D-A9FE-8A886C02BF71@mimectl> Message-ID: <488EBDCB.3000209@mellanox.co.il> > > Running rpmbuild --rebuild --define '_topdir /var/tmp/OFED_topdir' > > --define 'KVERSION 2.6.18-53.1.21.el5' --define '_release > > 2.6.18_53.1.21.el5' --define '_prefix /usr' > > /share/apps/OFED-1.3.1/SRPMS/ib-bonding-0.9.0-25.src.rpm > > ib-bonding-debuginfo was not created > > > > I then customised the installation to exclude ib-bonding-debuginfo, > > installation continued and failed to build any debuginfo RPMS. Finally I > > could complete the installation without any debuginfo libraries. > > > > Any idea what could I be doing wrong ??? > > > > Thank you, > > Amit > > > Make sure that you have "redhat-rpm-config" RPM installed. Regards, Vladimir From vlad at lists.openfabrics.org Tue Jul 29 02:55:00 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 29 Jul 2008 02:55:00 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080729-0200 daily build status Message-ID: <20080729095500.D6FF8E60327@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: From yevgenyp at mellanox.co.il Tue Jul 29 04:56:59 2008 From: yevgenyp at mellanox.co.il (Yevgeny Petrilin) Date: Tue, 29 Jul 2008 14:56:59 +0300 Subject: [ofa-general][PATCH] mlx4: Fixes to cqe format Message-ID: <488F058B.6040701@mellanox.co.il> mlx4: Fixes to cqe format Adjusted CQE format according to ConnectX PRM. Changes are required to enable mlx4_en driver to use the mlx4_cqe structure. Signed-off-by: Yevgeny Petrilin --- drivers/infiniband/hw/mlx4/cq.c | 33 ++++++++++++++++----------------- include/linux/mlx4/cq.h | 36 ++++++++++++++++++++++++------------ 2 files changed, 40 insertions(+), 29 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 0b191a4..fcfdd9e 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -514,17 +514,17 @@ static void mlx4_ib_handle_error_cqe(struct mlx4_err_cqe *cqe, wc->vendor_err = cqe->vendor_err_syndrome; } -static int mlx4_ib_ipoib_csum_ok(__be32 status, __be16 checksum) +static int mlx4_ib_ipoib_csum_ok(__be16 status, __be16 checksum) { - return ((status & cpu_to_be32(MLX4_CQE_IPOIB_STATUS_IPV4 | - MLX4_CQE_IPOIB_STATUS_IPV4F | - MLX4_CQE_IPOIB_STATUS_IPV4OPT | - MLX4_CQE_IPOIB_STATUS_IPV6 | - MLX4_CQE_IPOIB_STATUS_IPOK)) == - cpu_to_be32(MLX4_CQE_IPOIB_STATUS_IPV4 | - MLX4_CQE_IPOIB_STATUS_IPOK)) && - (status & cpu_to_be32(MLX4_CQE_IPOIB_STATUS_UDP | - MLX4_CQE_IPOIB_STATUS_TCP)) && + return ((status & cpu_to_be16(MLX4_CQE_STATUS_IPV4 | + MLX4_CQE_STATUS_IPV4F | + MLX4_CQE_STATUS_IPV4OPT | + MLX4_CQE_STATUS_IPV6 | + MLX4_CQE_STATUS_IPOK)) == + cpu_to_be16(MLX4_CQE_STATUS_IPV4 | + MLX4_CQE_STATUS_IPOK)) && + (status & cpu_to_be16(MLX4_CQE_STATUS_UDP | + MLX4_CQE_STATUS_TCP)) && checksum == cpu_to_be16(0xffff); } @@ -581,17 +581,17 @@ repoll: } if (!*cur_qp || - (be32_to_cpu(cqe->my_qpn) & 0xffffff) != (*cur_qp)->mqp.qpn) { + (be32_to_cpu(cqe->vlan_my_qpn) & MLX4_CQE_QPN_MASK) != (*cur_qp)->mqp.qpn) { /* * We do not have to take the QP table lock here, * because CQs will be locked while QPs are removed * from the table. */ mqp = __mlx4_qp_lookup(to_mdev(cq->ibcq.device)->dev, - be32_to_cpu(cqe->my_qpn)); + be32_to_cpu(cqe->vlan_my_qpn)); if (unlikely(!mqp)) { printk(KERN_WARNING "CQ %06x with entry for unknown QPN %06x\n", - cq->mcq.cqn, be32_to_cpu(cqe->my_qpn) & 0xffffff); + cq->mcq.cqn, be32_to_cpu(cqe->vlan_my_qpn) & MLX4_CQE_QPN_MASK); return -EINVAL; } @@ -691,14 +691,13 @@ repoll: } wc->slid = be16_to_cpu(cqe->rlid); - wc->sl = cqe->sl >> 4; + wc->sl = be16_to_cpu(cqe->sl_vid >> 12); g_mlpath_rqpn = be32_to_cpu(cqe->g_mlpath_rqpn); wc->src_qp = g_mlpath_rqpn & 0xffffff; wc->dlid_path_bits = (g_mlpath_rqpn >> 24) & 0x7f; wc->wc_flags |= g_mlpath_rqpn & 0x80000000 ? IB_WC_GRH : 0; wc->pkey_index = be32_to_cpu(cqe->immed_rss_invalid) & 0x7f; - wc->csum_ok = mlx4_ib_ipoib_csum_ok(cqe->ipoib_status, - cqe->checksum); + wc->csum_ok = mlx4_ib_ipoib_csum_ok(cqe->status, cqe->checksum); } return 0; @@ -766,7 +765,7 @@ void __mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq) */ while ((int) --prod_index - (int) cq->mcq.cons_index >= 0) { cqe = get_cqe(cq, prod_index & cq->ibcq.cqe); - if ((be32_to_cpu(cqe->my_qpn) & 0xffffff) == qpn) { + if ((be32_to_cpu(cqe->vlan_my_qpn) & MLX4_CQE_QPN_MASK) == qpn) { if (srq && !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) mlx4_ib_free_srq_wqe(srq, be16_to_cpu(cqe->wqe_index)); ++nfreed; diff --git a/include/linux/mlx4/cq.h b/include/linux/mlx4/cq.h index 071cf96..55023ca 100644 --- a/include/linux/mlx4/cq.h +++ b/include/linux/mlx4/cq.h @@ -39,17 +39,18 @@ #include struct mlx4_cqe { - __be32 my_qpn; + __be32 vlan_my_qpn; __be32 immed_rss_invalid; __be32 g_mlpath_rqpn; - u8 sl; - u8 reserved1; + __be16 sl_vid; __be16 rlid; - __be32 ipoib_status; + __be16 status; + u8 ipv6_mask; + u8 enc_bf; __be32 byte_cnt; __be16 wqe_index; __be16 checksum; - u8 reserved2[3]; + u8 reserved[3]; u8 owner_sr_opcode; }; @@ -64,6 +65,11 @@ struct mlx4_err_cqe { }; enum { + MLX4_CQE_VLAN_PRESENT_MASK = 1 << 29, + MLX4_CQE_QPN_MASK = 0xffffff, +}; + +enum { MLX4_CQE_OWNER_MASK = 0x80, MLX4_CQE_IS_SEND_MASK = 0x40, MLX4_CQE_OPCODE_MASK = 0x1f @@ -86,13 +92,19 @@ enum { }; enum { - MLX4_CQE_IPOIB_STATUS_IPV4 = 1 << 22, - MLX4_CQE_IPOIB_STATUS_IPV4F = 1 << 23, - MLX4_CQE_IPOIB_STATUS_IPV6 = 1 << 24, - MLX4_CQE_IPOIB_STATUS_IPV4OPT = 1 << 25, - MLX4_CQE_IPOIB_STATUS_TCP = 1 << 26, - MLX4_CQE_IPOIB_STATUS_UDP = 1 << 27, - MLX4_CQE_IPOIB_STATUS_IPOK = 1 << 28, + MLX4_CQE_STATUS_IPV4 = 1 << 6, + MLX4_CQE_STATUS_IPV4F = 1 << 7, + MLX4_CQE_STATUS_IPV6 = 1 << 8, + MLX4_CQE_STATUS_IPV4OPT = 1 << 9, + MLX4_CQE_STATUS_TCP = 1 << 10, + MLX4_CQE_STATUS_UDP = 1 << 11, + MLX4_CQE_STATUS_IPOK = 1 << 12, +}; + +enum { + MLX4_CQE_LLC = 1, + MLX4_CQE_SNAP = 1 << 1, + MLX4_CQE_BAD_FCS = 1 << 4, }; static inline void mlx4_cq_arm(struct mlx4_cq *cq, u32 cmd, -- 1.5.3.7 From tziporet at mellanox.co.il Tue Jul 29 07:10:59 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Tue, 29 Jul 2008 17:10:59 +0300 Subject: [ofa-general] OFED July 28, 2008 meeting minutes Message-ID: <5D49E7A8952DC44FB38C38FA0D758EAD30863B@mtlexch01.mtl.com> OFED meeting minutes for July 28, 2008 ====================================== 1. OFED 1.4 status: Need update on all open items: - MVAPICH 1.1 - should be ready next week - MVAPICH2 1.2 - should be ready next week - Open MPI 1.3 - work on some critical bug, hope to have first version in two weeks - NFSoRDMA - backport to RHEL5 and SLES 10 - Jeff Backer - no update yet - Beta release expected date: Since 2.6.27-rc1 was released today we expect the beta to be next week on Aug 7. 2. Testing description: Decided that each company that participate in qualifying OFED 1.4 will send a short report in the following format: - HW: HCA/RNIC - Switch/GW used - CPU arch - OSes - ULPs - Apps By having this we will be able to know our testing coverage (see Mellanox report below) Tziporet will collect all data and present the status in next meeting 3. Decided to add CentOS operating systems to be officially supported by OFED 1.4 Tziporet From hal.rosenstock at gmail.com Tue Jul 29 08:07:50 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 29 Jul 2008 11:07:50 -0400 Subject: [ofa-general] [PATCH] opensm/include/opensm/osm_subnet.h: Update some comments Message-ID: Sasha, See attached file (so hopefully not whitespace mangled) -- Hal -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-subneth1 Type: application/octet-stream Size: 728 bytes Desc: not available URL: From swise at opengridcomputing.com Tue Jul 29 08:17:10 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 29 Jul 2008 10:17:10 -0500 Subject: [ofa-general] Re: [RFC PATCH] rds: enable rdma on iWARP In-Reply-To: <488E0830.1050802@opengridcomputing.com> References: <20080728152919.GA16043@opengridcomputing.com> <200807281813.23845.olaf.kirch@oracle.com> <488E0830.1050802@opengridcomputing.com> Message-ID: <488F3476.9080202@opengridcomputing.com> Hey Olaf, So given we can post inv or fastreg on any connected QP in a PD, does this help the design for utilizing fastreg for RDS zcopy? From scott.king at ns.sympatico.ca Tue Jul 29 08:19:31 2008 From: scott.king at ns.sympatico.ca (BRITISH LOTTERY) Date: Tue, 29 Jul 2008 11:19:31 -0400 Subject: [ofa-general] Claims Requirements Message-ID: <20080729151931.YJRK1669.simmts6-srv.bellnexxia.net@simip10.srvr.bell.ca> Contact Mr.Edison Walker for the claim of £1,500,000 GBP which your I D won in BRITISH LOTTERY.Provide your Names,Address,Age,Occupation,Tel,Country send to:bnl_edison09 at yahoo.de From scott.king at ns.sympatico.ca Tue Jul 29 08:19:39 2008 From: scott.king at ns.sympatico.ca (BRITISH LOTTERY) Date: Tue, 29 Jul 2008 11:19:39 -0400 Subject: [ofa-general] Claims Requirements Message-ID: <20080729151939.EXRR1761.simmts8-srv.bellnexxia.net@simip11.srvr.bell.ca> Contact Mr.Edison Walker for the claim of £1,500,000 GBP which your I D won in BRITISH LOTTERY.Provide your Names,Address,Age,Occupation,Tel,Country send to:bnl_edison09 at yahoo.de From hal.rosenstock at gmail.com Tue Jul 29 08:28:42 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 29 Jul 2008 11:28:42 -0400 Subject: [ofa-general] [PATCH] opensm/osm_helper.c: Change lids_per_port to decimal Message-ID: Sasha, Please see attached file. -- Hal -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-helper1 Type: application/octet-stream Size: 1071 bytes Desc: not available URL: From hal.rosenstock at gmail.com Tue Jul 29 08:28:57 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 29 Jul 2008 11:28:57 -0400 Subject: [ofa-general] Re: opensm/osm_sa.c: GID format In-Reply-To: References: Message-ID: On Mon, Jul 28, 2008 at 10:07 AM, Hal Rosenstock wrote: > Should the GID format in opensm/osm_sa.c also be updated to use > inet_ntop/inet_pton rather than PRIx64 ? If so, should it also support > the old format for input so old files can still be imported ? Similarly, there's also the LID format in osm_db_pack.c in hex rather than decimal. -- Hal > > -- Hal > From PHF at zurich.ibm.com Tue Jul 29 08:32:40 2008 From: PHF at zurich.ibm.com (Philip Frey1) Date: Tue, 29 Jul 2008 17:32:40 +0200 Subject: [ofa-general] Non-Signaled RDMA Read Message-ID: Hello, I am trying to post a non-signaled RDMA Read WR. Somehow OFED 1.3 running on a T3 Chelsio Adapter seems not to care what I set for 'enum ibv_send_flags' in my 'struct ibv_send_wr'. I always get a work completion with status set to IBV_WC_SUCCESS and opcode IBV_WC_RDMA_READ. When creating the 'struct ibv_qp' I have set 'sq_sig_all = 0' in 'struct ibv_qp_init_attr'. With RDMA Write and Send I do not get a WC when I set the 'ibv_send_flags' to 0. What am I missing here? Many thanks, Philip -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Jul 29 08:43:22 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 29 Jul 2008 08:43:22 -0700 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: (Philip Frey1's message of "Tue, 29 Jul 2008 17:32:40 +0200") References: Message-ID: > I am trying to post a non-signaled RDMA Read WR. Somehow OFED 1.3 running > on a T3 Chelsio Adapter seems > not to care what I set for 'enum ibv_send_flags' in my 'struct > ibv_send_wr'. I always get a work completion with status > set to IBV_WC_SUCCESS and opcode IBV_WC_RDMA_READ. When creating the > 'struct ibv_qp' I have set > 'sq_sig_all = 0' in 'struct ibv_qp_init_attr'. With RDMA Write and Send I > do not get a WC when I set the 'ibv_send_flags' > to 0. > > What am I missing here? Looks like you're hitting this -- the cxgb3 driver has: case IB_WR_RDMA_READ: t3_wr_opcode = T3_WR_READ; t3_wr_flags = 0; /* T3 reads are always signaled */ so I guess the HW can't do unsignaled RDMA reads. - R. From hartlch14 at gmail.com Tue Jul 29 08:53:15 2008 From: hartlch14 at gmail.com (Chuck Hartley) Date: Tue, 29 Jul 2008 11:53:15 -0400 Subject: [ofa-general] RDMA IOV segment limit is always zero? Message-ID: When I call dat_ep_query(), the max IOV segments for RDMA read/write always is returned as zero. I tried it on a couple different adapters (Mellanox 26428 and 25204). Does the value zero mean something else, like "unlimited"? I'm using DAT 2.0 and the code looks like this: DAT_EP_PARAM ep_p; memset(&ep_p, 0, sizeof(ep_p)); DAT_RETURN ret = dat_ep_query (h_ep, DAT_EP_FIELD_ALL, &ep_p); And here is what I get: EP attributes: IA_handle: 0x634160 EP state: DAT_EP_STATE_UNCONNECTED Local IA address: 0x6393b0 Local port qualifier: 43973 Remote IA address: 0x635be0 Remote port qualifier: 0 PZ handle: 0x634670 Receive EVD handle: 0x635710 Request EVD handle: 0x635040 Connect EVD handle: 0x634bc0 SRQ handle: (nil) Service type: DAT_SERVICE_TYPE_RC Max message size: 65536 Max RDMA size: 65536 QOS: DAT_QOS_BEST_EFFORT Receive completion flags: DAT_COMPLETION_DEFAULT_FLAG Request completion flags: DAT_COMPLETION_DEFAULT_FLAG Max recv DTOs per SRQ: 6 Max request DTOs per SRQ: 7 Max recv IOV: 4 Max request IOV: 4 Max RDMA read in: 4 Max RDMA read out: 4 SRQ soft highwater: -1 Max IOV segments per RDMA read: 0 Max IOV segments per RDMA write: 0 Num transport attr: 0 Num provider attr: 0 Is there some utility to look at HCA parameters? Thanks, Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From swise at opengridcomputing.com Tue Jul 29 09:07:30 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 29 Jul 2008 11:07:30 -0500 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: References: Message-ID: <488F4042.4010108@opengridcomputing.com> Roland Dreier wrote: > > I am trying to post a non-signaled RDMA Read WR. Somehow OFED 1.3 running > > on a T3 Chelsio Adapter seems > > not to care what I set for 'enum ibv_send_flags' in my 'struct > > ibv_send_wr'. I always get a work completion with status > > set to IBV_WC_SUCCESS and opcode IBV_WC_RDMA_READ. When creating the > > 'struct ibv_qp' I have set > > 'sq_sig_all = 0' in 'struct ibv_qp_init_attr'. With RDMA Write and Send I > > do not get a WC when I set the 'ibv_send_flags' > > to 0. > > > > What am I missing here? > > Looks like you're hitting this -- the cxgb3 driver has: > > case IB_WR_RDMA_READ: > t3_wr_opcode = T3_WR_READ; > t3_wr_flags = 0; /* T3 reads are always signaled */ > > so I guess the HW can't do unsignaled RDMA reads. > > - R. > Right. This is a T3 hw limitation. It will be fixed in the next chip. Steve. From hal.rosenstock at gmail.com Tue Jul 29 09:16:08 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 29 Jul 2008 12:16:08 -0400 Subject: [ofa-general] [PATCH] OpenSM: More conversion to (unsiged) decimal lid format Message-ID: Sasha, Please see attached file. -- Hal -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-declid1 Type: application/octet-stream Size: 5535 bytes Desc: not available URL: From swise at opengridcomputing.com Tue Jul 29 09:32:27 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 29 Jul 2008 11:32:27 -0500 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <488F4042.4010108@opengridcomputing.com> References: <488F4042.4010108@opengridcomputing.com> Message-ID: <488F461B.5060602@opengridcomputing.com> Steve Wise wrote: > Roland Dreier wrote: >> > I am trying to post a non-signaled RDMA Read WR. Somehow OFED 1.3 >> running > on a T3 Chelsio Adapter seems >> > not to care what I set for 'enum ibv_send_flags' in my 'struct > >> ibv_send_wr'. I always get a work completion with status >> > set to IBV_WC_SUCCESS and opcode IBV_WC_RDMA_READ. When creating >> the > 'struct ibv_qp' I have set >> > 'sq_sig_all = 0' in 'struct ibv_qp_init_attr'. With RDMA Write and >> Send I > do not get a WC when I set the 'ibv_send_flags' >> > to 0. >> > > What am I missing here? >> >> Looks like you're hitting this -- the cxgb3 driver has: >> >> case IB_WR_RDMA_READ: >> t3_wr_opcode = T3_WR_READ; >> t3_wr_flags = 0; /* T3 reads are always >> signaled */ >> >> so I guess the HW can't do unsignaled RDMA reads. >> >> - R. >> > Right. This is a T3 hw limitation. It will be fixed in the next chip. > > Steve. > > Note that I _could_ add code in the driver/lib poll routine to "eat" these cqes. But there is a fundamental problem in that if the CQ is armed, then the HW _will_ generate an interrupt in this case. This leads to a "false notification" which breaks the semantics of the notification mechanism. I'm not sure how bad that is? To the application, it would receive a notification, then go poll and get nothing. Steve. From rdreier at cisco.com Tue Jul 29 09:37:26 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 29 Jul 2008 09:37:26 -0700 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <488F461B.5060602@opengridcomputing.com> (Steve Wise's message of "Tue, 29 Jul 2008 11:32:27 -0500") References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> Message-ID: > Note that I _could_ add code in the driver/lib poll routine to "eat" > these cqes. But there is a fundamental problem in that if the CQ is > armed, then the HW _will_ generate an interrupt in this case. This > leads to a "false notification" which breaks the semantics of the > notification mechanism. I'm not sure how bad that is? To the > application, it would receive a notification, then go poll and get > nothing. There's also the issue that these completions take CQ entries... so if an app provisions a CQ for some amount of completions and then assumes that posting 100 RDMA reads doesn't use any CQ entries, things break. Not sure if there is a good way to deal with this quirk really. - R. From arlin.r.davis at intel.com Tue Jul 29 09:43:38 2008 From: arlin.r.davis at intel.com (Davis, Arlin R) Date: Tue, 29 Jul 2008 09:43:38 -0700 Subject: [ofa-general] RDMA IOV segment limit is always zero? In-Reply-To: References: Message-ID: ________________________________ From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Chuck Hartley Sent: Tuesday, July 29, 2008 8:53 AM To: general at lists.openfabrics.org Subject: [ofa-general] RDMA IOV segment limit is always zero? When I call dat_ep_query(), the max IOV segments for RDMA read/write always is returned as zero. I tried it on a couple different adapters (Mellanox 26428 and 25204). Does the value zero mean something else, like "unlimited"? I'm using DAT 2.0 and the code looks like this: Which provder are you using? Is this the new socket cm provider? looks like the max_iov setting per rdma is not being set with the new socket cm provider. I will fix this for OFED 1.4. You can get the device attributes with "ibv_devinfo -v" -arlin -------------- next part -------------- An HTML attachment was scrubbed... URL: From felix at chelsio.com Tue Jul 29 09:47:00 2008 From: felix at chelsio.com (Felix Marti) Date: Tue, 29 Jul 2008 09:47:00 -0700 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) References: <488F4042.4010108@opengridcomputing.com><488F461B.5060602@opengridcomputing.com> Message-ID: <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> > -----Original Message----- > From: general-bounces at lists.openfabrics.org [mailto:general- > bounces at lists.openfabrics.org] On Behalf Of Roland Dreier > Sent: Tuesday, July 29, 2008 9:37 AM > To: Steve Wise > Cc: Philip Frey1; general at lists.openfabrics.org > Subject: Re: [ofa-general] Non-Signaled RDMA Read (cxgb3) > > > Note that I _could_ add code in the driver/lib poll routine to "eat" > > these cqes. But there is a fundamental problem in that if the CQ is > > armed, then the HW _will_ generate an interrupt in this case. This > > leads to a "false notification" which breaks the semantics of the > > notification mechanism. I'm not sure how bad that is? To the > > application, it would receive a notification, then go poll and get > > nothing. > > There's also the issue that these completions take CQ entries... so if > an app provisions a CQ for some amount of completions and then assumes > that posting 100 RDMA reads doesn't use any CQ entries, things break. > > Not sure if there is a good way to deal with this quirk really. Roland, AFAIK, even an unsignaled WR generates a CQE when an error occurs (and then all subsequent WRs complete in error as well). Doesn't that imply that the CQ must be sized assuming that _every_ WR can generate a CQE? > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib- > general From PHF at zurich.ibm.com Tue Jul 29 09:53:59 2008 From: PHF at zurich.ibm.com (Philip Frey1) Date: Tue, 29 Jul 2008 18:53:59 +0200 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <488F461B.5060602@opengridcomputing.com> References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> Message-ID: general-bounces at lists.openfabrics.org wrote on 29.07.2008 18:32:27: > [image removed] > > Re: [ofa-general] Non-Signaled RDMA Read (cxgb3) > > Steve Wise > > to: > > Roland Dreier > > 29.07.2008 18:32 > > Sent by: > > general-bounces at lists.openfabrics.org > > Cc: > > Philip Frey1, general > > Steve Wise wrote: > > Roland Dreier wrote: > >> > I am trying to post a non-signaled RDMA Read WR. Somehow OFED 1.3 > >> running > on a T3 Chelsio Adapter seems > >> > not to care what I set for 'enum ibv_send_flags' in my 'struct > > >> ibv_send_wr'. I always get a work completion with status > >> > set to IBV_WC_SUCCESS and opcode IBV_WC_RDMA_READ. When creating > >> the > 'struct ibv_qp' I have set > >> > 'sq_sig_all = 0' in 'struct ibv_qp_init_attr'. With RDMA Write and > >> Send I > do not get a WC when I set the 'ibv_send_flags' > >> > to 0. > >> > > What am I missing here? > >> > >> Looks like you're hitting this -- the cxgb3 driver has: > >> > >> case IB_WR_RDMA_READ: > >> t3_wr_opcode = T3_WR_READ; > >> t3_wr_flags = 0; /* T3 reads are always > >> signaled */ > >> > >> so I guess the HW can't do unsignaled RDMA reads. > >> > >> - R. > >> > > Right. This is a T3 hw limitation. It will be fixed in the next chip. > > > > Steve. > > > > > Note that I _could_ add code in the driver/lib poll routine to "eat" > these cqes. But there is a fundamental problem in that if the CQ is > armed, then the HW _will_ generate an interrupt in this case. This > leads to a "false notification" which breaks the semantics of the > notification mechanism. I'm not sure how bad that is? To the > application, it would receive a notification, then go poll and get nothing. > > Steve. I do not think this will help. When I do a send, I am expecting a completion with the matching work request ID and I do not want to add extra code into the upper layer app to get rid of those false notifications (pitfall). Speaking about this notification mechanism: What is the intended way of using that? Say I expect a number of inbound sends. What disturbs me is that only the first of them generates an event. I cannot just do an ibv_get_cq_event() after having gotten the first one. It might happen that already several inbound sends have arrived so I need to poll the CQ for all of them. Only when the polling fails I go to sleep waiting for an event. From a user perspective it would be way more convenient to just call ibv_get_cq_event() several times. Another disturbing thing is the fact that the events have to be acknowledged - why bother the user with that? If I may, I would like to suggest another thing: Is there no possibility that non-signaled WRs do not create a WC at all? Clearing the SQ with a signaled WR to prevent an overflow is rather cumbersome as the user has to keep track of the number of unsignaled WRs posted as well as the size of the SQ. Things like that add a lot of pitfalls to upper layer applications. Thanks, Philip -------------- next part -------------- An HTML attachment was scrubbed... URL: From swise at opengridcomputing.com Tue Jul 29 09:59:51 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 29 Jul 2008 11:59:51 -0500 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> Message-ID: <488F4C87.5020004@opengridcomputing.com> Roland Dreier wrote: > > Note that I _could_ add code in the driver/lib poll routine to "eat" > > these cqes. But there is a fundamental problem in that if the CQ is > > armed, then the HW _will_ generate an interrupt in this case. This > > leads to a "false notification" which breaks the semantics of the > > notification mechanism. I'm not sure how bad that is? To the > > application, it would receive a notification, then go poll and get > > nothing. > > There's also the issue that these completions take CQ entries... so if > an app provisions a CQ for some amount of completions and then assumes > that posting 100 RDMA reads doesn't use any CQ entries, things break. > > Note such an app could overflow the cq if the reads fail. Granted that causes the connection to come crashing down anyway so maybe its ok. > Not sure if there is a good way to deal with this quirk really. > > I tend to agree. That's why I haven't done it. :) Stevo From alekseys at voltaire.com Tue Jul 29 10:11:01 2008 From: alekseys at voltaire.com (Aleksey Senin) Date: Tue, 29 Jul 2008 20:11:01 +0300 Subject: [ofa-general][RFC] IPv6 support for RDMA CM Message-ID: <1217351461.15603.24.camel@linux-zn6t.site> Patch adds IPv6 support for RDMA CM operations. It provides basic support and have limitations bellow: No support for link-local addresses. Can't use mixed IPv6 and IPv4 addresses as source and destinations Not perform checks against ANYCAST address type. Details: struct addr_req extended to support sockaddr_in6 family Functions like address_resolve_local, addr_resolve_remote changed to get pointer to generic sockaddr struct insted of sockadd_in. Such functions used as upper layer, and after parsing sa_family call corresponding IPv6 ot IPv4 function. To perform network discovery and to symbols should be exported: nd_table ndisc_send_ns The points that must be improved to specified above are: recognition of local device by given IP address sending network discovery without exporting symbols Waiting for feedback, suggestions. ----------------- diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c index 09a2bec..3c3f665 100644 --- a/drivers/infiniband/core/addr.c +++ b/drivers/infiniband/core/addr.c @@ -38,10 +38,13 @@ #include #include #include +#include +#include #include #include #include #include +#include MODULE_AUTHOR("Sean Hefty"); MODULE_DESCRIPTION("IB Address Translation"); @@ -50,7 +53,9 @@ MODULE_LICENSE("Dual BSD/GPL"); struct addr_req { struct list_head list; struct sockaddr src_addr; + u8 src_pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; struct sockaddr dst_addr; + u8 dst_pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; struct rdma_dev_addr *addr; struct rdma_addr_client *client; void *context; @@ -113,15 +118,24 @@ EXPORT_SYMBOL(rdma_copy_addr); int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr) { struct net_device *dev; - __be32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr; int ret; - dev = ip_dev_find(&init_net, ip); - if (!dev) + if (addr->sa_family == AF_INET ) { + __be32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr; + dev = ip_dev_find(&init_net, ip); + if (!dev) return -EADDRNOTAVAIL; - ret = rdma_copy_addr(dev_addr, dev, NULL); - dev_put(dev); + ret = rdma_copy_addr(dev_addr, dev, NULL); + dev_put(dev); + } else { + for_each_netdev(&init_net, dev) + if ( ipv6_chk_addr(&init_net, &((struct sockaddr_in6*)addr)->sin6_addr, dev, 1) ) { + ret = rdma_copy_addr(dev_addr, dev, NULL); + break; + } + } + return ret; } EXPORT_SYMBOL(rdma_translate_ip); @@ -171,7 +185,8 @@ static void addr_send_arp(struct sockaddr_in *dst_in) ip_rt_put(rt); } -static int addr_resolve_remote(struct sockaddr_in *src_in, + +static int addr4_resolve_remote(struct sockaddr_in *src_in, struct sockaddr_in *dst_in, struct rdma_dev_addr *addr) { @@ -220,10 +235,78 @@ out: return ret; } +static int addr6_resolve_remote(struct sockaddr_in6 *src_in, + struct sockaddr_in6 *dst_in, + struct rdma_dev_addr *addr) +{ + + struct neighbour *neigh; + int ret=-ENODATA; + struct dst_entry *dst; + struct in6_addr *target; + struct in6_addr mcaddr; + + struct flowi fl = { + .nl_u = { + .ip6_u = { + .daddr = dst_in->sin6_addr, + .saddr = src_in->sin6_addr, + }, + }, + }; + + dst = ip6_route_output(&init_net, NULL, &fl); + if(!dst) + goto out; + + /* If the device does ARP internally, return 'done' */ + if (dst->dev->flags & IFF_NOARP) { + ret = rdma_copy_addr(addr, dst->dev, NULL); + goto put; + } + + neigh = ndisc_get_neigh(dst->dev, &dst_in->sin6_addr); + if (!neigh) { + ret = -ENOMEM; + goto put; + } + + if (!(neigh->nud_state & NUD_VALID)) { + target = (struct in6_addr*)&neigh->primary_key; + addrconf_addr_solict_mult(target, &mcaddr); + ndisc_send_ns(dst->dev, neigh, target, &mcaddr , NULL); + /* ret = rdma_copy_addr(addr, neigh->dev, neigh->ha); */ + + ret = -ENODATA; + goto release; + } + ret = rdma_copy_addr(addr, neigh->dev, neigh->ha); + + +release: + neigh_release(neigh); +put: + dst_release(dst); +out: + return ret; +} + +static int addr_resolve_remote(struct sockaddr *src_in, + struct sockaddr *dst_in, + struct rdma_dev_addr *addr) +{ + int ret = -ENODATA; + if ( src_in->sa_family == AF_INET ) { + ret = addr4_resolve_remote((struct sockaddr_in*)src_in, (struct sockaddr_in*)dst_in, addr); + } else if ( src_in->sa_family == AF_INET6 ) { + ret = addr6_resolve_remote((struct sockaddr_in6*)src_in, (struct sockaddr_in6*)dst_in, addr); + } + return ret; +} + static void process_req(struct work_struct *work) { struct addr_req *req, *temp_req; - struct sockaddr_in *src_in, *dst_in; struct list_head done_list; INIT_LIST_HEAD(&done_list); @@ -231,10 +314,7 @@ static void process_req(struct work_struct *work) mutex_lock(&lock); list_for_each_entry_safe(req, temp_req, &req_list, list) { if (req->status == -ENODATA) { - src_in = (struct sockaddr_in *) &req->src_addr; - dst_in = (struct sockaddr_in *) &req->dst_addr; - req->status = addr_resolve_remote(src_in, dst_in, - req->addr); + req->status = addr_resolve_remote(&req->src_addr, &req->dst_addr, req->addr); if (req->status && time_after_eq(jiffies, req->timeout)) req->status = -ETIMEDOUT; else if (req->status == -ENODATA) @@ -258,7 +338,7 @@ static void process_req(struct work_struct *work) } } -static int addr_resolve_local(struct sockaddr_in *src_in, +static int addr4_resolve_local(struct sockaddr_in *src_in, struct sockaddr_in *dst_in, struct rdma_dev_addr *addr) { @@ -289,6 +369,24 @@ static int addr_resolve_local(struct sockaddr_in *src_in, return ret; } +static int addr6_resolve_local(struct sockaddr_in6 *src_in, + struct sockaddr_in6 *dst_in, + struct rdma_dev_addr *addr) +{ + return -EADDRNOTAVAIL; +} + +static int addr_resolve_local(struct sockaddr *src_in, + struct sockaddr *dst_in, + struct rdma_dev_addr *addr) +{ + if ( src_in->sa_family == AF_INET ) { + return addr4_resolve_local((struct sockaddr_in*)src_in, (struct sockaddr_in*)dst_in, addr); + } else + return addr6_resolve_local((struct sockaddr_in6*)src_in, (struct sockaddr_in6*)dst_in, addr); +} + + int rdma_resolve_ip(struct rdma_addr_client *client, struct sockaddr *src_addr, struct sockaddr *dst_addr, struct rdma_dev_addr *addr, int timeout_ms, @@ -296,7 +394,6 @@ int rdma_resolve_ip(struct rdma_addr_client *client, struct rdma_dev_addr *addr, void *context), void *context) { - struct sockaddr_in *src_in, *dst_in; struct addr_req *req; int ret = 0; @@ -313,12 +410,10 @@ int rdma_resolve_ip(struct rdma_addr_client *client, req->client = client; atomic_inc(&client->refcount); - src_in = (struct sockaddr_in *) &req->src_addr; - dst_in = (struct sockaddr_in *) &req->dst_addr; - - req->status = addr_resolve_local(src_in, dst_in, addr); - if (req->status == -EADDRNOTAVAIL) - req->status = addr_resolve_remote(src_in, dst_in, addr); + req->status = addr_resolve_local(&req->src_addr, &req->dst_addr, addr); + if (req->status == -EADDRNOTAVAIL) { + req->status = addr_resolve_remote(&req->src_addr, &req->dst_addr, addr); + } switch (req->status) { case 0: @@ -328,7 +423,8 @@ int rdma_resolve_ip(struct rdma_addr_client *client, case -ENODATA: req->timeout = msecs_to_jiffies(timeout_ms) + jiffies; queue_req(req); - addr_send_arp(dst_in); + if(req->dst_addr.sa_family == AF_INET ) + addr_send_arp((struct sockaddr_in*)&req->dst_addr); break; default: ret = req->status; diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index e980ff3..87ae995 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -2074,7 +2074,7 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr) struct rdma_id_private *id_priv; int ret; - if (addr->sa_family != AF_INET) + if (addr->sa_family != AF_INET && addr->sa_family != AF_INET6 ) return -EAFNOSUPPORT; id_priv = container_of(id, struct rdma_id_private, id); @@ -2115,30 +2115,58 @@ static int cma_format_hdr(void *hdr, enum rdma_port_space ps, struct rdma_route *route) { struct sockaddr_in *src4, *dst4; + struct sockaddr_in6 *src6, *dst6; struct cma_hdr *cma_hdr; struct sdp_hh *sdp_hdr; - - src4 = (struct sockaddr_in *) &route->addr.src_addr; - dst4 = (struct sockaddr_in *) &route->addr.dst_addr; - - switch (ps) { - case RDMA_PS_SDP: - sdp_hdr = hdr; - if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) - return -EINVAL; - sdp_set_ip_ver(sdp_hdr, 4); - sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; - sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; - sdp_hdr->port = src4->sin_port; - break; - default: - cma_hdr = hdr; - cma_hdr->cma_version = CMA_VERSION; - cma_set_ip_ver(cma_hdr, 4); - cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; - cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; - cma_hdr->port = src4->sin_port; - break; + if ( route->addr.src_addr.sa_family == AF_INET) { + src4 = (struct sockaddr_in *) &route->addr.src_addr; + dst4 = (struct sockaddr_in *) &route->addr.dst_addr; + }else if (route->addr.src_addr.sa_family == AF_INET6) { + src6 = (struct sockaddr_in6 *) &route->addr.src_addr; + dst6 = (struct sockaddr_in6 *) &route->addr.dst_addr; + }else + return -EINVAL; + + if ( route->addr.src_addr.sa_family == AF_INET) { + switch (ps) { + case RDMA_PS_SDP: + sdp_hdr = hdr; + if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) + return -EINVAL; + sdp_set_ip_ver(sdp_hdr, 4); + sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; + sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; + sdp_hdr->port = src4->sin_port; + break; + default: + cma_hdr = hdr; + cma_hdr->cma_version = CMA_VERSION; + cma_set_ip_ver(cma_hdr, 4); + cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; + cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; + cma_hdr->port = src4->sin_port; + break; + } + }else { + switch (ps) { + case RDMA_PS_SDP: + sdp_hdr = hdr; + if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) + return -EINVAL; + sdp_set_ip_ver(sdp_hdr, 6); + sdp_hdr->src_addr.ip6 = src6->sin6_addr; + sdp_hdr->dst_addr.ip6 = dst6->sin6_addr; + sdp_hdr->port = src6->sin6_port; + break; + default: + cma_hdr = hdr; + cma_hdr->cma_version = CMA_VERSION; + cma_set_ip_ver(cma_hdr, 6); + cma_hdr->src_addr.ip6 = src6->sin6_addr; + cma_hdr->dst_addr.ip6 = dst6->sin6_addr; + cma_hdr->port = src6->sin6_port; + break; + } } return 0; } diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index beb48e3..fb1f59c 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -157,6 +157,7 @@ struct neigh_table nd_tbl = { .gc_thresh3 = 1024, }; +EXPORT_SYMBOL(nd_tbl); /* ND options */ struct ndisc_options { struct nd_opt_hdr *nd_opt_array[__ND_OPT_ARRAY_MAX]; @@ -586,6 +587,8 @@ void ndisc_send_ns(struct net_device *dev, struct neighbour *neigh, !ipv6_addr_any(saddr) ? ND_OPT_SOURCE_LL_ADDR : 0); } +EXPORT_SYMBOL(ndisc_send_ns); + void ndisc_send_rs(struct net_device *dev, const struct in6_addr *saddr, const struct in6_addr *daddr) { -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Jul 29 10:19:44 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 29 Jul 2008 10:19:44 -0700 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> (Felix Marti's message of "Tue, 29 Jul 2008 09:47:00 -0700") References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> Message-ID: > Roland, AFAIK, even an unsignaled WR generates a CQE when an error > occurs (and then all subsequent WRs complete in error as well). Doesn't > that imply that the CQ must be sized assuming that _every_ WR can > generate a CQE? good point, I forgot about that. From caitlin.bestler at neterion.com Tue Jul 29 10:20:17 2008 From: caitlin.bestler at neterion.com (Caitlin Bestler) Date: Tue, 29 Jul 2008 10:20:17 -0700 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> Message-ID: <469958e00807291020q33ff5b34j847066640283dff9@mail.gmail.com> On Tue, Jul 29, 2008 at 9:47 AM, Felix Marti wrote: > > > Roland, AFAIK, even an unsignaled WR generates a CQE when an error > occurs (and then all subsequent WRs complete in error as well). Doesn't > that imply that the CQ must be sized assuming that _every_ WR can > generate a CQE? > Yes, that is exactly what application writers should assume. They should also assume that spurious completion interrupts are possible (although obviously they should be minimized) and therefore the application should always be ready be woken up early and find nothing to do. More complete descriptions that could be used as guidelines can be found in the DAPL manuals. About the only thing I think that can be done about this specific quirk is to document it so that application developers are aware of it, and know that it is a model-specific limitation and do not think it is a limitation of RDMA or iWARP. If their application does not need the notification it should keep requesting unsignaled completion. From rdreier at cisco.com Tue Jul 29 10:26:52 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 29 Jul 2008 10:26:52 -0700 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: (Philip Frey1's message of "Tue, 29 Jul 2008 18:53:59 +0200") References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> Message-ID: > I do not think this will help. When I do a send, I am expecting a > completion with the matching work request ID and I do not want to add > extra code into the upper layer app to get rid of those false > notifications (pitfall). Your app would just get woken up, poll the CQ, and see that the CQ was empty. So it would presumably just go back to sleep until the next event. > Speaking about this notification mechanism: What is the intended way > of using that? Say I expect a number of inbound sends. What disturbs > me is that only the first of them generates an event. I cannot just > do an ibv_get_cq_event() after having gotten the first one. It might > happen that already several inbound sends have arrived so I need to > poll the CQ for all of them. Only when the polling fails I go to > sleep waiting for an event. From a user perspective it would be way > more convenient to just call ibv_get_cq_event() several times. CQ notification events are distinct from CQ entries. events are fairly heavyweight to create -- they involve servicing an interrupt from the device and scheduling the right process to receive the event. CQ entries are cheap -- the adapter just DMAs a data structure into memory, and the application can read it when it's ready. > Another disturbing thing is the fact that the events have to be > acknowledged - why bother the user with that? There are races around destroying CQs and receiving events for CQs that have already been destroyed that can't (AFAICS) be fixed without the acknowledgement. Note that you don't have to ack events until just before you destroy the CQ. > If I may, I would like to suggest another thing: Is there no > possibility that non-signaled WRs do not create a WC at all? Clearing > the SQ with a signaled WR to prevent an overflow is rather cumbersome > as the user has to keep track of the number of unsignaled WRs posted > as well as the size of the SQ. Things like that add a lot of pitfalls > to upper layer applications. If there are no completions ever then there is no way for the driver to know when a send queue slot can be reused, and no way for the app to know when resources can be released either. - R. From dotanba at gmail.com Wed Jul 30 11:44:42 2008 From: dotanba at gmail.com (Dotan Barak) Date: Wed, 30 Jul 2008 20:44:42 +0200 Subject: [ofa-general] Non-Signaled RDMA Read In-Reply-To: References: Message-ID: <4890B69A.9060100@gmail.com> Philip Frey1 wrote: > > Hello, > > I am trying to post a non-signaled RDMA Read WR. Somehow OFED 1.3 > running on a T3 Chelsio Adapter seems > not to care what I set for 'enum ibv_send_flags' in my 'struct > ibv_send_wr'. I always get a work completion with status > set to IBV_WC_SUCCESS and opcode IBV_WC_RDMA_READ. When creating the > 'struct ibv_qp' I have set > 'sq_sig_all = 0' in 'struct ibv_qp_init_attr'. With RDMA Write and > Send I do not get a WC when I set the 'ibv_send_flags' > to 0. > > What am I missing here? You do everything as you should, the same test case is working as expected in other low level driver Maybe it is a low level driver issue ... Dotan > > Many thanks, > Philip > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From chu11 at llnl.gov Tue Jul 29 10:47:13 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 29 Jul 2008 10:47:13 -0700 Subject: [ofa-general] Re: [OpenSM][Trivial] Change --ucast_file to --ucast_dump_file In-Reply-To: <20080728224831.GC5487@sashak.voltaire.com> References: <1217278759.16508.360.camel@cardanus.llnl.gov> <20080728224831.GC5487@sashak.voltaire.com> Message-ID: <1217353633.16508.373.camel@cardanus.llnl.gov> Hey Sasha, lfts_file would be fine as well. I just wanted to make it consistent throughout. Patch is attached. Note, I tweaked a lot of documentation because of this. Wherever it stated "dump file", I changed to "lfts file" or just "file" as appropriate. Al On Tue, 2008-07-29 at 01:48 +0300, Sasha Khapyorsky wrote: > Hi Al, > > On 13:59 Mon 28 Jul , Al Chu wrote: > > > > Simple patch. Just change the command line option for consistency to > > the config file, variable name, etc. > > > > Alternately, we could rename everything else from ucast_dump_file to > > ucast_file. > > I think it would be better because this is not a dump file, but *input* > LFTs file. Probably it is even better to rename it to 'lfts_file' or so. > What do you think? > > Sasha -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-rename-ucast_file-and-ucast_dump_file-to-lft_file.patch Type: text/x-patch Size: 10763 bytes Desc: not available URL: From chu11 at llnl.gov Tue Jul 29 10:48:21 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 29 Jul 2008 10:48:21 -0700 Subject: [ofa-general] [OpenSM][Trivial] Fix instruction typo in config file Message-ID: <1217353701.16508.376.camel@cardanus.llnl.gov> Hey Sasha, Nothing fancy. Just noticed this typo. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-fix-config-file-instruction-typo.patch Type: text/x-patch Size: 964 bytes Desc: not available URL: From swise at opengridcomputing.com Tue Jul 29 12:45:27 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 29 Jul 2008 14:45:27 -0500 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <469958e00807291020q33ff5b34j847066640283dff9@mail.gmail.com> References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> <469958e00807291020q33ff5b34j847066640283dff9@mail.gmail.com> Message-ID: <488F7357.9090004@opengridcomputing.com> Caitlin Bestler wrote: > On Tue, Jul 29, 2008 at 9:47 AM, Felix Marti wrote: > > > >> Roland, AFAIK, even an unsignaled WR generates a CQE when an error >> occurs (and then all subsequent WRs complete in error as well). Doesn't >> that imply that the CQ must be sized assuming that _every_ WR can >> generate a CQE? >> >> > > Yes, that is exactly what application writers should assume. > > They should also assume that spurious completion interrupts are possible > (although obviously they should be minimized) and therefore the application > should always be ready be woken up early and find nothing to do. > > More complete descriptions that could be used as guidelines can be found > in the DAPL manuals. > > About the only thing I think that can be done about this specific quirk is > to document it so that application developers are aware of it, and know that > it is a model-specific limitation and do not think it is a limitation of RDMA > or iWARP. If their application does not need the notification it should > keep requesting unsignaled completion. > Well if applications should handle infrequent spurious notifcations where no cqe is available, then I can indeed add code in my provider poll to silently throw away unsignaled read completions. Steve. From sashak at voltaire.com Tue Jul 29 15:56:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 01:56:08 +0300 Subject: [ofa-general] Re: [OpenSM][Trivial] Change --ucast_file to --ucast_dump_file In-Reply-To: <1217353633.16508.373.camel@cardanus.llnl.gov> References: <1217278759.16508.360.camel@cardanus.llnl.gov> <20080728224831.GC5487@sashak.voltaire.com> <1217353633.16508.373.camel@cardanus.llnl.gov> Message-ID: <20080729225608.GH5487@sashak.voltaire.com> On 10:47 Tue 29 Jul , Al Chu wrote: > Subject: [PATCH] rename ucast_file and ucast_dump_file to lft_file > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 29 16:09:47 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 02:09:47 +0300 Subject: [ofa-general] Re: [OpenSM][Trivial] Fix instruction typo in config file In-Reply-To: <1217353701.16508.376.camel@cardanus.llnl.gov> References: <1217353701.16508.376.camel@cardanus.llnl.gov> Message-ID: <20080729230947.GJ5487@sashak.voltaire.com> On 10:48 Tue 29 Jul , Al Chu wrote: > Subject: [PATCH] fix config file instruction typo > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 29 16:23:13 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 02:23:13 +0300 Subject: [ofa-general] Re: [PATCH] OpenSM: More conversion to (unsiged) decimal lid format In-Reply-To: References: Message-ID: <20080729232313.GK5487@sashak.voltaire.com> On 12:16 Tue 29 Jul , Hal Rosenstock wrote: > > Please see attached file. All three are applied. Thanks. Sasha From sashak at voltaire.com Tue Jul 29 16:27:58 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 02:27:58 +0300 Subject: [ofa-general] Re: opensm/osm_sa.c: GID format In-Reply-To: References: Message-ID: <20080729232758.GL5487@sashak.voltaire.com> On 11:28 Tue 29 Jul , Hal Rosenstock wrote: > On Mon, Jul 28, 2008 at 10:07 AM, Hal Rosenstock > wrote: > > Should the GID format in opensm/osm_sa.c also be updated to use > > inet_ntop/inet_pton rather than PRIx64 ? If so, should it also support > > the old format for input so old files can still be imported ? > > Similarly, there's also the LID format in osm_db_pack.c in hex rather > than decimal. I think it could be converted. Parser uses strtoul(str, NULL, 0) to read this and should work fine with hex and decimal numbers. Sasha From sashak at voltaire.com Tue Jul 29 16:29:30 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 02:29:30 +0300 Subject: [ofa-general] Re: opensm/osm_sa.c: GID format In-Reply-To: References: Message-ID: <20080729232930.GM5487@sashak.voltaire.com> On 10:07 Mon 28 Jul , Hal Rosenstock wrote: > Should the GID format in opensm/osm_sa.c also be updated to use > inet_ntop/inet_pton rather than PRIx64 ? I have nothing against this. > If so, should it also support > the old format for input so old files can still be imported ? Seems like an overkill for me. Sasha From iuzzolin at nmia.com Wed Jul 30 01:32:58 2008 From: iuzzolin at nmia.com (Harold/Carlyn Iuzzolino) Date: Wed, 30 Jul 2008 02:32:58 -0600 Subject: [ofa-general] Under which version of Fedora does the current source of OFED compile? Message-ID: <200807300832.m6U8WwC15947@gandalf.iuzzolino.com> Dear Openfabrics (in particular Roland Dreier), general at lists.openfabrics.org Re: my problems with ipath_dma.c and/or dma-mapping.h not compiling in version OFED-1.4-20080724-0600.tgz under Fedora 6. > > It's a broken backport. valid_dma_direction() was moved into generic > code between 2.6.18 and 2.6.19, so someone needs to add a backport patch > to handle kernels 2.6.18 and older. > R. If that is true, under which version of Fedora does OFED-1.4-20080724-0600.tgz compile correctly? By looking at the postings on the general mailing list, I noticed that you are working with kernels 2.6.26 and 2.6.27. Fedora 8's kernel is "only" 2.6.21. I tried compiling OFED-1.4-20080724-0600.tgz on Fedora 8 and ran into 3 problems, two of which I figured out a workaround and the third problem I haven't found a solution. ---------------------------- Problem 1 Fedora 8's rpm build has a script /usr/lib/rpm/check-buildroot that DIDN'T exist in Fedora 6 and it causes the compile to fail because of a problem in the install.pl script and the rpm program /usr/lib/rpm/check-buildroot. -- It fails with the complaint from /usr/lib/rpm/check-buildroot: Found '/var/tmp/OFED' in installed files; aborting -- Turns out that complaint has TWO causes. The posting http://lists.openfabrics.org/pipermail/general/2008-March/047320.html to the general mailing list has a good explanation of the problem. And he filed this as Bug 968. This problem was reported on March 1st, FIVE months ago, and the bug is still in the install script. The program check-buildroot looks to see whether the compiled files in directory, $TOPDIR=/var/tmp/OFED_topdir, and its subdirectories, contains the string of $RPM_BUILD_ROOT=/var/tmp/OFED I fixed it by changing the install.pl script original line: my $TOPDIR = "/var/tmp/" . $PACKAGE . "_topdir"; fixed line: my $TOPDIR = "/var/tmp/" . "kernelbuild." . $PACKAGE . "_topdir"; so that the topdir is now /var/tmp/kernelbuild.OFED_topdir instead of /var/tmp/OFED_topdir The compile should fail for EVERYBODY that is using the Fedora 8 version of rpm 'build'. I don't understand why the team hasn't run into this problem as soon as you tried compiling on Fedora 8. Since you are up to kernel 2.6.26, you must have tried it on Fedora 8 which is kernel 2.6.21. That solved part of the problem but not all. PROBLEM 2 In the file /var/tmp/OFED/usr/src/ofa_kernel-1.3.2/ofed_scripts/ofa_kernel.spec there is the line BuildRoot: %{?build_root:%{build_root}}%{!?build_root:/var/tmp/OFED} so obviously that line would ALWAYS fail the test no matter what TOPDIR was. I changed the system routine /usr/lib/rpm/check-buildroot. Original lines: find "$RPM_BUILD_ROOT" \! \( \ -name '*.pyo' -o -name '*.pyc' -o -name '*.elc' -o -name '.packlist' \ Changed lines: find "$RPM_BUILD_ROOT" \! \( \ -name '*.spec' -o -name '*.pyo' -o -name '*.pyc' -o -name '*.elc' -o -name '.packlist' \ This bug will catch you if you are using Fedora 8 but not Fedora 6 because the rpm version of FC-6 doesn't have check-buildroot. FC-8's rpm build does have that program. --------------------------------------------------- PROBLEM 3 The installation script requires tcl-devel version 8.4.15 to match the rpm tcl-8.4.15-5.fc8 that comes with Fedora 8. I couldn't find tcl-devel-8.4.15.rpm anywhere. It doesn't come with Fedora 8 and I couldn't find it on the rpmfind website either. How did Openfabrics get Infiniband to compile on Fedora 8? Will I have any better luck with Fedora 9 which just came out? Carlyn Iuzzolino From vlad at lists.openfabrics.org Wed Jul 30 02:54:56 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Wed, 30 Jul 2008 02:54:56 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080730-0200 daily build status Message-ID: <20080730095456.29966E60C69@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: From Phillip at infoelephant.com Wed Jul 30 03:54:43 2008 From: Phillip at infoelephant.com (Phillip Mitchell) Date: Wed, 30 Jul 2008 14:54:43 +0400 Subject: [ofa-general] to Eddie Message-ID: <13dc01c8f232$ac206570$4f6f2774@Phillip> The guy picks out a plug an eyeball! Video! See Now! http://drustvovezi.org/clk/new.php -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Wed Jul 30 04:18:42 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 14:18:42 +0300 Subject: [ofa-general] [PATCH] opensm/include/Makefile.am: don't duplicate header files in EXTRA_DIST Message-ID: <20080730111842.GC16811@sashak.voltaire.com> Some header file names are listed in various Makefile.am in *HEADRES variables and also duplicated in EXTRA_DIST variable in include/Makefile.am. This patch removes such header file names from EXTRA_DIST list. Signed-off-by: Sasha Khapyorsky --- opensm/include/Makefile.am | 48 -------------------------------------------- 1 files changed, 0 insertions(+), 48 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index 48264ff..010cc7a 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -7,17 +7,14 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_lid_mgr.h \ $(srcdir)/opensm/osm_port.h \ $(srcdir)/opensm/osm_rand_fwd_tbl.h \ - $(srcdir)/opensm/osm_madw.h \ $(srcdir)/opensm/osm_subnet.h \ $(srcdir)/opensm/osm_sweep_fail_ctrl.h \ $(srcdir)/opensm/osm_partition.h \ - $(srcdir)/opensm/osm_helper.h \ $(srcdir)/opensm/osm_node.h \ $(srcdir)/opensm/osm_console.h \ $(srcdir)/opensm/osm_console_io.h \ $(srcdir)/opensm/osm_mcm_info.h \ $(srcdir)/opensm/osm_inform.h \ - $(srcdir)/opensm/osm_path.h \ $(srcdir)/opensm/osm_service.h \ $(srcdir)/opensm/osm_switch.h \ $(srcdir)/opensm/osm_router.h \ @@ -26,24 +23,19 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_sa.h \ $(srcdir)/opensm/osm_port_profile.h \ $(srcdir)/opensm/osm_multicast.h \ - $(srcdir)/opensm/osm_base.h \ $(srcdir)/opensm/osm_errors.h \ - $(srcdir)/opensm/osm_event_plugin.h \ $(srcdir)/opensm/osm_mtree.h \ $(srcdir)/opensm/osm_sm.h \ $(srcdir)/opensm/osm_lin_fwd_tbl.h \ $(srcdir)/opensm/osm_ucast_mgr.h \ $(srcdir)/opensm/osm_db.h \ - $(srcdir)/opensm/osm_mad_pool.h \ $(srcdir)/opensm/osm_remote_sm.h \ - $(srcdir)/opensm/osm_msgdef.h \ $(srcdir)/opensm/st.h \ $(srcdir)/opensm/osm_mcast_tbl.h \ $(srcdir)/opensm/osm_pkey.h \ $(srcdir)/opensm/osm_pkey_mgr.h \ $(srcdir)/opensm/osm_sa_mad_ctrl.h \ $(srcdir)/opensm/osm_mcm_port.h \ - $(srcdir)/opensm/osm_log.h \ $(srcdir)/opensm/osm_fwd_tbl.h \ $(srcdir)/opensm/osm_db_pack.h \ $(srcdir)/opensm/osm_sm_mad_ctrl.h \ @@ -54,42 +46,6 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_perfmgr_db.h \ $(srcdir)/opensm/osm_qos_policy.h \ $(srcdir)/opensm/osm_qos_parser_y.h \ - $(srcdir)/complib/cl_thread_osd.h \ - $(srcdir)/complib/cl_packon.h \ - $(srcdir)/complib/cl_atomic_osd.h \ - $(srcdir)/complib/cl_spinlock.h \ - $(srcdir)/complib/cl_passivelock.h \ - $(srcdir)/complib/cl_timer_osd.h \ - $(srcdir)/complib/cl_debug.h \ - $(srcdir)/complib/cl_event_osd.h \ - $(srcdir)/complib/cl_ptr_vector.h \ - $(srcdir)/complib/cl_comppool.h \ - $(srcdir)/complib/cl_math.h \ - $(srcdir)/complib/cl_qpool.h \ - $(srcdir)/complib/cl_qlist.h \ - $(srcdir)/complib/cl_vector.h \ - $(srcdir)/complib/cl_byteswap_osd.h \ - $(srcdir)/complib/cl_event_wheel.h \ - $(srcdir)/complib/cl_thread.h \ - $(srcdir)/complib/cl_packoff.h \ - $(srcdir)/complib/cl_pool.h \ - $(srcdir)/complib/cl_types_osd.h \ - $(srcdir)/complib/cl_list.h \ - $(srcdir)/complib/cl_atomic.h \ - $(srcdir)/complib/cl_map.h \ - $(srcdir)/complib/cl_timer.h \ - $(srcdir)/complib/cl_event.h \ - $(srcdir)/complib/cl_log.h \ - $(srcdir)/complib/cl_dispatcher.h \ - $(srcdir)/complib/cl_spinlock_osd.h \ - $(srcdir)/complib/cl_debug_osd.h \ - $(srcdir)/complib/cl_qmap.h \ - $(srcdir)/complib/cl_byteswap.h \ - $(srcdir)/complib/cl_threadpool.h \ - $(srcdir)/complib/cl_types.h \ - $(srcdir)/complib/cl_fleximap.h \ - $(srcdir)/complib/cl_qcomppool.h \ - $(srcdir)/complib/cl_nodenamemap.h \ $(srcdir)/iba/ib_types.h \ $(srcdir)/iba/ib_cm_types.h \ $(srcdir)/vendor/osm_vendor_mlx_transport_anafa.h \ @@ -100,9 +56,7 @@ EXTRA_DIST = \ $(srcdir)/vendor/osm_vendor_mtl_transaction_mgr.h \ $(srcdir)/vendor/osm_vendor_mlx_sar.h \ $(srcdir)/vendor/osm_vendor_mlx_dispatcher.h \ - $(srcdir)/vendor/osm_vendor_api.h \ $(srcdir)/vendor/osm_vendor_umadt.h \ - $(srcdir)/vendor/osm_pkt_randomizer.h \ $(srcdir)/vendor/osm_vendor_mlx_svc.h \ $(srcdir)/vendor/osm_vendor_mlx_hca.h \ $(srcdir)/vendor/osm_vendor_mlx_rmpp_ctx.h \ @@ -111,12 +65,10 @@ EXTRA_DIST = \ $(srcdir)/vendor/osm_vendor_mtl_hca_guid.h \ $(srcdir)/vendor/osm_vendor_select.h \ $(srcdir)/vendor/osm_vendor_test.h \ - $(srcdir)/vendor/osm_vendor.h \ $(srcdir)/vendor/osm_vendor_ts.h \ $(srcdir)/vendor/osm_vendor_mlx_txn.h \ $(srcdir)/vendor/osm_vendor_al.h \ $(srcdir)/vendor/osm_vendor_mtl.h \ - $(srcdir)/vendor/osm_vendor_sa_api.h \ $(srcdir)/vendor/osm_ts_useraccess.h \ $(srcdir)/vendor/osm_umadt.h \ $(srcdir)/vendor/osm_mtl_bind.h -- 1.5.5.1.178.g1f811 From sashak at voltaire.com Wed Jul 30 04:20:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 14:20:05 +0300 Subject: [ofa-general] [PATCH] opensm: install all OpenSM header files In-Reply-To: <20080730111842.GC16811@sashak.voltaire.com> References: <20080730111842.GC16811@sashak.voltaire.com> Message-ID: <20080730112005.GD16811@sashak.voltaire.com> Install (publish) all OpenSM header files - this is useful for plugin builders. Signed-off-by: Sasha Khapyorsky --- opensm/include/Makefile.am | 42 ------------------------------ opensm/opensm/Makefile.am | 60 ++++++++++++++++++++++++++++++++++++++------ 2 files changed, 52 insertions(+), 50 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index 010cc7a..f1b4504 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -4,48 +4,6 @@ SUBDIRS = . nobase_pkginclude_HEADERS = iba/ib_types.h iba/ib_cm_types.h EXTRA_DIST = \ - $(srcdir)/opensm/osm_lid_mgr.h \ - $(srcdir)/opensm/osm_port.h \ - $(srcdir)/opensm/osm_rand_fwd_tbl.h \ - $(srcdir)/opensm/osm_subnet.h \ - $(srcdir)/opensm/osm_sweep_fail_ctrl.h \ - $(srcdir)/opensm/osm_partition.h \ - $(srcdir)/opensm/osm_node.h \ - $(srcdir)/opensm/osm_console.h \ - $(srcdir)/opensm/osm_console_io.h \ - $(srcdir)/opensm/osm_mcm_info.h \ - $(srcdir)/opensm/osm_inform.h \ - $(srcdir)/opensm/osm_service.h \ - $(srcdir)/opensm/osm_switch.h \ - $(srcdir)/opensm/osm_router.h \ - $(srcdir)/opensm/osm_prefix_route.h \ - $(srcdir)/opensm/osm_opensm.h \ - $(srcdir)/opensm/osm_sa.h \ - $(srcdir)/opensm/osm_port_profile.h \ - $(srcdir)/opensm/osm_multicast.h \ - $(srcdir)/opensm/osm_errors.h \ - $(srcdir)/opensm/osm_mtree.h \ - $(srcdir)/opensm/osm_sm.h \ - $(srcdir)/opensm/osm_lin_fwd_tbl.h \ - $(srcdir)/opensm/osm_ucast_mgr.h \ - $(srcdir)/opensm/osm_db.h \ - $(srcdir)/opensm/osm_remote_sm.h \ - $(srcdir)/opensm/st.h \ - $(srcdir)/opensm/osm_mcast_tbl.h \ - $(srcdir)/opensm/osm_pkey.h \ - $(srcdir)/opensm/osm_pkey_mgr.h \ - $(srcdir)/opensm/osm_sa_mad_ctrl.h \ - $(srcdir)/opensm/osm_mcm_port.h \ - $(srcdir)/opensm/osm_fwd_tbl.h \ - $(srcdir)/opensm/osm_db_pack.h \ - $(srcdir)/opensm/osm_sm_mad_ctrl.h \ - $(srcdir)/opensm/osm_attrib_req.h \ - $(srcdir)/opensm/osm_stats.h \ - $(srcdir)/opensm/osm_vl15intf.h \ - $(srcdir)/opensm/osm_perfmgr.h \ - $(srcdir)/opensm/osm_perfmgr_db.h \ - $(srcdir)/opensm/osm_qos_policy.h \ - $(srcdir)/opensm/osm_qos_parser_y.h \ $(srcdir)/iba/ib_types.h \ $(srcdir)/iba/ib_cm_types.h \ $(srcdir)/vendor/osm_vendor_mlx_transport_anafa.h \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 0ea562d..0974cac 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -91,14 +91,58 @@ opensm_LDADD = -L../complib -losmcomp -L../libvendor -losmvendor -L. -lopensm $( opensmincludedir = $(includedir)/infiniband/opensm -opensminclude_HEADERS = $(srcdir)/../include/opensm/osm_base.h \ - $(srcdir)/../include/opensm/osm_event_plugin.h \ - $(srcdir)/../include/opensm/osm_helper.h \ - $(srcdir)/../include/opensm/osm_log.h \ - $(srcdir)/../include/opensm/osm_mad_pool.h \ - $(srcdir)/../include/opensm/osm_madw.h \ - $(srcdir)/../include/opensm/osm_msgdef.h \ - $(srcdir)/../include/opensm/osm_path.h +opensminclude_HEADERS = \ + $(srcdir)/../include/opensm/osm_attrib_req.h \ + $(srcdir)/../include/opensm/osm_base.h \ + $(srcdir)/../include/opensm/osm_console.h \ + $(srcdir)/../include/opensm/osm_console_io.h \ + $(srcdir)/../include/opensm/osm_db.h \ + $(srcdir)/../include/opensm/osm_db_pack.h \ + $(srcdir)/../include/opensm/osm_event_plugin.h \ + $(srcdir)/../include/opensm/osm_errors.h \ + $(srcdir)/../include/opensm/osm_fwd_tbl.h \ + $(srcdir)/../include/opensm/osm_helper.h \ + $(srcdir)/../include/opensm/osm_inform.h \ + $(srcdir)/../include/opensm/osm_lid_mgr.h \ + $(srcdir)/../include/opensm/osm_lin_fwd_tbl.h \ + $(srcdir)/../include/opensm/osm_log.h \ + $(srcdir)/../include/opensm/osm_mad_pool.h \ + $(srcdir)/../include/opensm/osm_madw.h \ + $(srcdir)/../include/opensm/osm_mcast_tbl.h \ + $(srcdir)/../include/opensm/osm_mcm_info.h \ + $(srcdir)/../include/opensm/osm_mcm_port.h \ + $(srcdir)/../include/opensm/osm_mtree.h \ + $(srcdir)/../include/opensm/osm_multicast.h \ + $(srcdir)/../include/opensm/osm_msgdef.h \ + $(srcdir)/../include/opensm/osm_node.h \ + $(srcdir)/../include/opensm/osm_opensm.h \ + $(srcdir)/../include/opensm/osm_partition.h \ + $(srcdir)/../include/opensm/osm_path.h \ + $(srcdir)/../include/opensm/osm_perfmgr.h \ + $(srcdir)/../include/opensm/osm_perfmgr_db.h \ + $(srcdir)/../include/opensm/osm_pkey.h \ + $(srcdir)/../include/opensm/osm_pkey_mgr.h \ + $(srcdir)/../include/opensm/osm_port.h \ + $(srcdir)/../include/opensm/osm_port_profile.h \ + $(srcdir)/../include/opensm/osm_prefix_route.h \ + $(srcdir)/../include/opensm/osm_qos_parser_y.h \ + $(srcdir)/../include/opensm/osm_qos_policy.h \ + $(srcdir)/../include/opensm/osm_rand_fwd_tbl.h \ + $(srcdir)/../include/opensm/osm_remote_sm.h \ + $(srcdir)/../include/opensm/osm_router.h \ + $(srcdir)/../include/opensm/osm_sa.h \ + $(srcdir)/../include/opensm/osm_sa_mad_ctrl.h \ + $(srcdir)/../include/opensm/osm_service.h \ + $(srcdir)/../include/opensm/osm_sm.h \ + $(srcdir)/../include/opensm/osm_sm_mad_ctrl.h \ + $(srcdir)/../include/opensm/st.h \ + $(srcdir)/../include/opensm/osm_stats.h \ + $(srcdir)/../include/opensm/osm_subnet.h \ + $(srcdir)/../include/opensm/osm_sweep_fail_ctrl.h \ + $(srcdir)/../include/opensm/osm_switch.h \ + $(srcdir)/../include/opensm/osm_ucast_mgr.h \ + $(srcdir)/../include/opensm/osm_vl15intf.h \ + $(builddir)/../include/opensm/osm_version.h BUILT_SOURCES = osm_version osm_version: -- 1.5.5.1.178.g1f811 From alekseys at voltaire.com Wed Jul 30 04:22:04 2008 From: alekseys at voltaire.com (Aleksey Senin) Date: Wed, 30 Jul 2008 14:22:04 +0300 Subject: [ofa-general][RFC] IPv6 support for RDMA CM Message-ID: <1217416924.32115.9.camel@linux-zn6t.site> Here is better formatted version of patch. Patch adds IPv6 support for RDMA CM operations. It provides basic support and have limitations bellow: No support for link-local addresses. Can't use mixed IPv6 and IPv4 addresses as source and destinations Not perform checks against ANYCAST address type. Details: struct addr_req extended to support sockaddr_in6 family Functions like address_resolve_local, addr_resolve_remote changed to get pointer to generic sockaddr struct insted of sockadd_in. Such functions used as upper layer, and after parsing sa_family call corresponding IPv6 ot IPv4 function. To perform network discovery and to symbols should be exported: nd_table ndisc_send_ns The points that must be improved to specified above are: recognition of local device by given IP address sending network discovery without exporting symbols Waiting for feedback, suggestions. ------------ drivers/infiniband/core/addr.c | 158 +++++++++++++++++++++++++++------- drivers/infiniband/core/cma.c | 72 +++++++++++++----- net/ipv6/ndisc.c | 2 3 files changed, 186 insertions(+), 46 deletions(-) diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c index 09a2bec..05ab1d0 100644 --- a/drivers/infiniband/core/addr.c +++ b/drivers/infiniband/core/addr.c @@ -38,10 +38,13 @@ #include #include #include +#include +#include #include #include #include #include +#include MODULE_AUTHOR("Sean Hefty"); MODULE_DESCRIPTION("IB Address Translation"); @@ -50,7 +53,9 @@ MODULE_LICENSE("Dual BSD/GPL"); struct addr_req { struct list_head list; struct sockaddr src_addr; + u8 src_pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; struct sockaddr dst_addr; + u8 dst_pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; struct rdma_dev_addr *addr; struct rdma_addr_client *client; void *context; @@ -113,15 +118,27 @@ EXPORT_SYMBOL(rdma_copy_addr); int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr) { struct net_device *dev; - __be32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr; int ret; - dev = ip_dev_find(&init_net, ip); - if (!dev) - return -EADDRNOTAVAIL; + if (addr->sa_family == AF_INET) { + __be32 ip = ((struct sockaddr_in *)addr)->sin_addr.s_addr; + dev = ip_dev_find(&init_net, ip); + + if (!dev) + return -EADDRNOTAVAIL; + + ret = rdma_copy_addr(dev_addr, dev, NULL); + dev_put(dev); + } else { + struct in6_addr *i = &((struct sockaddr_in6 *)addr)->sin6_addr; + for_each_netdev(&init_net, dev) { + if (ipv6_chk_addr(&init_net, i , dev, 1)) { + ret = rdma_copy_addr(dev_addr, dev, NULL); + break; + } + } + } - ret = rdma_copy_addr(dev_addr, dev, NULL); - dev_put(dev); return ret; } EXPORT_SYMBOL(rdma_translate_ip); @@ -171,7 +188,8 @@ static void addr_send_arp(struct sockaddr_in *dst_in) ip_rt_put(rt); } -static int addr_resolve_remote(struct sockaddr_in *src_in, + +static int addr4_resolve_remote(struct sockaddr_in *src_in, struct sockaddr_in *dst_in, struct rdma_dev_addr *addr) { @@ -220,10 +238,78 @@ out: return ret; } +static int addr6_resolve_remote(struct sockaddr_in6 *src_in, + struct sockaddr_in6 *dst_in, + struct rdma_dev_addr *addr) +{ + + struct neighbour *neigh; + int ret = -ENODATA; + struct dst_entry *dst; + struct in6_addr *target; + struct in6_addr mcaddr; + + struct flowi fl = { + .nl_u = { + .ip6_u = { + .daddr = dst_in->sin6_addr, + .saddr = src_in->sin6_addr, + }, + }, + }; + + dst = ip6_route_output(&init_net, NULL, &fl); + if (!dst) + goto out; + + /* If the device does ARP internally, return 'done' */ + if (dst->dev->flags & IFF_NOARP) { + ret = rdma_copy_addr(addr, dst->dev, NULL); + goto put; + } + + neigh = ndisc_get_neigh(dst->dev, &dst_in->sin6_addr); + if (!neigh) { + ret = -ENOMEM; + goto put; + } + + if (!(neigh->nud_state & NUD_VALID)) { + target = (struct in6_addr *)&neigh->primary_key; + addrconf_addr_solict_mult(target, &mcaddr); + ndisc_send_ns(dst->dev, neigh, target, &mcaddr , NULL); + ret = -ENODATA; + goto release; + } + ret = rdma_copy_addr(addr, neigh->dev, neigh->ha); + + +release: + neigh_release(neigh); +put: + dst_release(dst); +out: + return ret; +} + +static int addr_resolve_remote(struct sockaddr *src_in, + struct sockaddr *dst_in, + struct rdma_dev_addr *addr) +{ + int ret = -ENODATA; + if (src_in->sa_family == AF_INET) { + ret = addr4_resolve_remote((struct sockaddr_in *)src_in, \ + (struct sockaddr_in *)dst_in, addr); + } else if (src_in->sa_family == AF_INET6) { + ret = addr6_resolve_remote((struct sockaddr_in6 *)src_in, \ + (struct sockaddr_in6 *)dst_in, addr); + } + return ret; +} + static void process_req(struct work_struct *work) { struct addr_req *req, *temp_req; - struct sockaddr_in *src_in, *dst_in; struct list_head done_list; INIT_LIST_HEAD(&done_list); @@ -231,14 +317,13 @@ static void process_req(struct work_struct *work) mutex_lock(&lock); list_for_each_entry_safe(req, temp_req, &req_list, list) { if (req->status == -ENODATA) { - src_in = (struct sockaddr_in *) &req->src_addr; - dst_in = (struct sockaddr_in *) &req->dst_addr; - req->status = addr_resolve_remote(src_in, dst_in, - req->addr); - if (req->status && time_after_eq(jiffies, req->timeout)) - req->status = -ETIMEDOUT; - else if (req->status == -ENODATA) - continue; + req->status = addr_resolve_remote(&req->src_addr, \ + &req->dst_addr, req->addr); + + if (req->status && time_after_eq(jiffies, req->timeout)) + req->status = -ETIMEDOUT; + else if (req->status == -ENODATA) + continue; } list_move_tail(&req->list, &done_list); } @@ -258,7 +343,7 @@ static void process_req(struct work_struct *work) } } -static int addr_resolve_local(struct sockaddr_in *src_in, +static int addr4_resolve_local(struct sockaddr_in *src_in, struct sockaddr_in *dst_in, struct rdma_dev_addr *addr) { @@ -289,6 +374,30 @@ static int addr_resolve_local(struct sockaddr_in *src_in, return ret; } +static int addr6_resolve_local(struct sockaddr_in6 *src_in, + struct sockaddr_in6 *dst_in, + struct rdma_dev_addr *addr) +{ + return -EADDRNOTAVAIL; +} + +static int addr_resolve_local(struct sockaddr *src_in, + struct sockaddr *dst_in, + struct rdma_dev_addr *addr) +{ + int ret; + + if (src_in->sa_family == AF_INET) { + ret = addr4_resolve_local((struct sockaddr_in *)src_in, \ + (struct sockaddr_in *)dst_in, addr); + } else { + ret = addr6_resolve_local((struct sockaddr_in6 *)src_in, \ + (struct sockaddr_in6 *)dst_in, addr); + } + return ret; +} + + int rdma_resolve_ip(struct rdma_addr_client *client, struct sockaddr *src_addr, struct sockaddr *dst_addr, struct rdma_dev_addr *addr, int timeout_ms, @@ -296,7 +405,6 @@ int rdma_resolve_ip(struct rdma_addr_client *client, struct rdma_dev_addr *addr, void *context), void *context) { - struct sockaddr_in *src_in, *dst_in; struct addr_req *req; int ret = 0; @@ -313,12 +421,11 @@ int rdma_resolve_ip(struct rdma_addr_client *client, req->client = client; atomic_inc(&client->refcount); - src_in = (struct sockaddr_in *) &req->src_addr; - dst_in = (struct sockaddr_in *) &req->dst_addr; - - req->status = addr_resolve_local(src_in, dst_in, addr); - if (req->status == -EADDRNOTAVAIL) - req->status = addr_resolve_remote(src_in, dst_in, addr); + req->status = addr_resolve_local(&req->src_addr, &req->dst_addr, addr); + if (req->status == -EADDRNOTAVAIL) { + req->status = addr_resolve_remote(&req->src_addr, \ + &req->dst_addr, addr); + } switch (req->status) { case 0: @@ -328,7 +435,8 @@ int rdma_resolve_ip(struct rdma_addr_client *client, case -ENODATA: req->timeout = msecs_to_jiffies(timeout_ms) + jiffies; queue_req(req); - addr_send_arp(dst_in); + if (req->dst_addr.sa_family == AF_INET) + addr_send_arp((struct sockaddr_in *)&req->dst_addr); break; default: ret = req->status; diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index e980ff3..0a1a413 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -2074,7 +2074,7 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr) struct rdma_id_private *id_priv; int ret; - if (addr->sa_family != AF_INET) + if (addr->sa_family != AF_INET && addr->sa_family != AF_INET6) return -EAFNOSUPPORT; id_priv = container_of(id, struct rdma_id_private, id); @@ -2115,30 +2115,60 @@ static int cma_format_hdr(void *hdr, enum rdma_port_space ps, struct rdma_route *route) { struct sockaddr_in *src4, *dst4; + struct sockaddr_in6 *src6, *dst6; struct cma_hdr *cma_hdr; struct sdp_hh *sdp_hdr; + if (route->addr.src_addr.sa_family == AF_INET) { + src4 = (struct sockaddr_in *) &route->addr.src_addr; + dst4 = (struct sockaddr_in *) &route->addr.dst_addr; + } else if (route->addr.src_addr.sa_family == AF_INET6) { + src6 = (struct sockaddr_in6 *) &route->addr.src_addr; + dst6 = (struct sockaddr_in6 *) &route->addr.dst_addr; + } else + return -EINVAL; - src4 = (struct sockaddr_in *) &route->addr.src_addr; - dst4 = (struct sockaddr_in *) &route->addr.dst_addr; + if (route->addr.src_addr.sa_family == AF_INET) { + switch (ps) { + case RDMA_PS_SDP: + sdp_hdr = hdr; - switch (ps) { - case RDMA_PS_SDP: - sdp_hdr = hdr; - if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) - return -EINVAL; - sdp_set_ip_ver(sdp_hdr, 4); - sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; - sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; - sdp_hdr->port = src4->sin_port; - break; - default: - cma_hdr = hdr; - cma_hdr->cma_version = CMA_VERSION; - cma_set_ip_ver(cma_hdr, 4); - cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; - cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; - cma_hdr->port = src4->sin_port; - break; + if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) + return -EINVAL; + + sdp_set_ip_ver(sdp_hdr, 4); + sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; + sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; + sdp_hdr->port = src4->sin_port; + break; + default: + cma_hdr = hdr; + cma_hdr->cma_version = CMA_VERSION; + cma_set_ip_ver(cma_hdr, 4); + cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; + cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; + cma_hdr->port = src4->sin_port; + break; + } + } else { + switch (ps) { + case RDMA_PS_SDP: + sdp_hdr = hdr; + if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) + return -EINVAL; + sdp_set_ip_ver(sdp_hdr, 6); + sdp_hdr->src_addr.ip6 = src6->sin6_addr; + sdp_hdr->dst_addr.ip6 = dst6->sin6_addr; + sdp_hdr->port = src6->sin6_port; + break; + default: + cma_hdr = hdr; + cma_hdr->cma_version = CMA_VERSION; + cma_set_ip_ver(cma_hdr, 6); + cma_hdr->src_addr.ip6 = src6->sin6_addr; + cma_hdr->dst_addr.ip6 = dst6->sin6_addr; + cma_hdr->port = src6->sin6_port; + break; + } } return 0; } diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index beb48e3..014edd6 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -156,6 +156,7 @@ struct neigh_table nd_tbl = { .gc_thresh2 = 512, .gc_thresh3 = 1024, }; +EXPORT_SYMBOL(nd_tbl); /* ND options */ struct ndisc_options { @@ -585,6 +586,7 @@ void ndisc_send_ns(struct net_device *dev, struct neighbour *neigh, &icmp6h, solicit, !ipv6_addr_any(saddr) ? ND_OPT_SOURCE_LL_ADDR : 0); } +EXPORT_SYMBOL(ndisc_send_ns); void ndisc_send_rs(struct net_device *dev, const struct in6_addr *saddr, const struct in6_addr *daddr) From sashak at voltaire.com Wed Jul 30 04:21:16 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 30 Jul 2008 14:21:16 +0300 Subject: [ofa-general] [PATCH] opensm/event_plugin: plugin API version 2 In-Reply-To: <20080730112005.GD16811@sashak.voltaire.com> References: <20080730111842.GC16811@sashak.voltaire.com> <20080730112005.GD16811@sashak.voltaire.com> Message-ID: <20080730112116.GE16811@sashak.voltaire.com> Main difference is that construct() method is renamed to create() and gets reference to osm_opensm_t object instead of just osm_log_t. This will provide to event plugin access to all OpenSM data structures and should help to create more generic plugins. For consistency with above destroy() method is renamed to delete(). Also OpenSM version is added there and plugin loader will check for exact match. Event plugin interface version is increased and osmeventplugin example updated accordingly. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_event_plugin.h | 14 +++++----- opensm/opensm/osm_event_plugin.c | 36 ++++++++++++--------------- opensm/opensm/osm_opensm.c | 2 +- opensm/osmeventplugin/src/osmeventplugin.c | 22 +++++++++++----- 4 files changed, 39 insertions(+), 35 deletions(-) diff --git a/opensm/include/opensm/osm_event_plugin.h b/opensm/include/opensm/osm_event_plugin.h index d9e546c..0626b86 100644 --- a/opensm/include/opensm/osm_event_plugin.h +++ b/opensm/include/opensm/osm_event_plugin.h @@ -37,7 +37,6 @@ #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -60,6 +59,8 @@ BEGIN_C_DECLS *********/ #define OSM_EPI_NODE_NAME_LEN (128) + +struct osm_opensm; /** ========================================================================= * Event types */ @@ -144,11 +145,11 @@ typedef struct osm_epi_trap_event { * The version should be set to OSM_EVENT_PLUGIN_INTERFACE_VER */ #define OSM_EVENT_PLUGIN_IMPL_NAME "osm_event_plugin" -#define OSM_EVENT_PLUGIN_INTERFACE_VER (1) +#define OSM_EVENT_PLUGIN_INTERFACE_VER 2 typedef struct osm_event_plugin { - int interface_version; - void *(*construct) (osm_log_t * osm_log); - void (*destroy) (void *plugin_data); + const char *osm_version; + void *(*create) (struct osm_opensm *osm); + void (*delete) (void *plugin_data); void (*report) (void *plugin_data, osm_epi_event_id_t event_id, void *event_data); } osm_event_plugin_t; @@ -161,14 +162,13 @@ typedef struct osm_epi_plugin { void *handle; osm_event_plugin_t *impl; void *plugin_data; - osm_log_t *p_log; char *plugin_name; } osm_epi_plugin_t; /** * functions */ -osm_epi_plugin_t *osm_epi_construct(osm_log_t * p_log, char *plugin_name); +osm_epi_plugin_t *osm_epi_construct(struct osm_opensm *osm, char *plugin_name); void osm_epi_destroy(osm_epi_plugin_t * plugin); /** ========================================================================= diff --git a/opensm/opensm/osm_event_plugin.c b/opensm/opensm/osm_event_plugin.c index e579fad..e07fd8d 100644 --- a/opensm/opensm/osm_event_plugin.c +++ b/opensm/opensm/osm_event_plugin.c @@ -44,8 +44,8 @@ #include #include - #include +#include #if defined(PATH_MAX) #define OSM_PATH_MAX (PATH_MAX + 1) @@ -58,7 +58,7 @@ /** * functions */ -osm_epi_plugin_t *osm_epi_construct(osm_log_t * p_log, char *plugin_name) +osm_epi_plugin_t *osm_epi_construct(osm_opensm_t *osm, char *plugin_name) { char lib_name[OSM_PATH_MAX]; osm_epi_plugin_t *rc = NULL; @@ -75,7 +75,7 @@ osm_epi_plugin_t *osm_epi_construct(osm_log_t * p_log, char *plugin_name) rc->handle = dlopen(lib_name, RTLD_LAZY); if (!rc->handle) { - OSM_LOG(p_log, OSM_LOG_ERROR, + OSM_LOG(&osm->log, OSM_LOG_ERROR, "Failed to open event plugin \"%s\" : \"%s\"\n", lib_name, dlerror()); goto DLOPENFAIL; @@ -85,37 +85,33 @@ osm_epi_plugin_t *osm_epi_construct(osm_log_t * p_log, char *plugin_name) (osm_event_plugin_t *) dlsym(rc->handle, OSM_EVENT_PLUGIN_IMPL_NAME); if (!rc->impl) { - OSM_LOG(p_log, OSM_LOG_ERROR, + OSM_LOG(&osm->log, OSM_LOG_ERROR, "Failed to find \"%s\" symbol in \"%s\" : \"%s\"\n", OSM_EVENT_PLUGIN_IMPL_NAME, lib_name, dlerror()); goto Exit; } /* Check the version to make sure this module will work with us */ - if (rc->impl->interface_version != OSM_EVENT_PLUGIN_INTERFACE_VER) { - OSM_LOG(p_log, OSM_LOG_ERROR, - "Error opening %s: " - "%s symbol is the wrong version %d != %d\n", - plugin_name, - OSM_EVENT_PLUGIN_IMPL_NAME, - rc->impl->interface_version, - OSM_EVENT_PLUGIN_INTERFACE_VER); + if (strcmp(rc->impl->osm_version, osm->osm_version)) { + OSM_LOG(&osm->log, OSM_LOG_ERROR, "Error loading plugin" + " \'%s\': OpenSM version mismatch - plugin was built" + " against %s version of OpenSM. Skip loading.\n", + plugin_name, rc->impl->osm_version); goto Exit; } - if (!rc->impl->construct) { - OSM_LOG(p_log, OSM_LOG_ERROR, - "%s symbol has no construct function\n", - OSM_EVENT_PLUGIN_IMPL_NAME); + if (!rc->impl->create) { + OSM_LOG(&osm->log, OSM_LOG_ERROR, + "Error loading plugin \'%s\': no create() method.\n", + plugin_name); goto Exit; } - rc->plugin_data = rc->impl->construct(p_log); + rc->plugin_data = rc->impl->create(osm); if (!rc->plugin_data) goto Exit; - rc->p_log = p_log; rc->plugin_name = strdup(plugin_name); return (rc); @@ -129,8 +125,8 @@ DLOPENFAIL: void osm_epi_destroy(osm_epi_plugin_t * plugin) { if (plugin) { - if (plugin->impl->destroy) - plugin->impl->destroy(plugin->plugin_data); + if (plugin->impl->delete) + plugin->impl->delete(plugin->plugin_data); dlclose(plugin->handle); free(plugin->plugin_name); free(plugin); diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c index 0101b97..6cf4726 100644 --- a/opensm/opensm/osm_opensm.c +++ b/opensm/opensm/osm_opensm.c @@ -251,7 +251,7 @@ static void load_plugins(osm_opensm_t *osm, char *plugin_names) name = strtok_r(plugin_names, " \t\n", &p); while (name && *name) { - epi = osm_epi_construct(&osm->log, name); + epi = osm_epi_construct(osm, name); if (!epi) osm_log(&osm->log, OSM_LOG_ERROR, "cannot load plugin \'%s\'\n", name); diff --git a/opensm/osmeventplugin/src/osmeventplugin.c b/opensm/osmeventplugin/src/osmeventplugin.c index 6cc4c70..74e37d5 100644 --- a/opensm/osmeventplugin/src/osmeventplugin.c +++ b/opensm/osmeventplugin/src/osmeventplugin.c @@ -38,12 +38,15 @@ #include #include #include +#include #include #include #include -#include #include #include +#include +#include +#include /** ========================================================================= * This is a simple example plugin which logs some of the events the OSM @@ -57,7 +60,7 @@ typedef struct _log_events { /** ========================================================================= */ -static void *construct(osm_log_t * osmlog) +static void *construct(osm_opensm_t *osm) { _log_events_t *log = malloc(sizeof(*log)); if (!log) @@ -66,14 +69,14 @@ static void *construct(osm_log_t * osmlog) log->log_file = fopen(SAMPLE_PLUGIN_OUTPUT_FILE, "a+"); if (!(log->log_file)) { - osm_log(osmlog, OSM_LOG_ERROR, + osm_log(&osm->log, OSM_LOG_ERROR, "Sample Event Plugin: Failed to open output file \"%s\"\n", SAMPLE_PLUGIN_OUTPUT_FILE); free(log); return (NULL); } - log->osmlog = osmlog; + log->osmlog = &osm->log; return ((void *)log); } @@ -171,9 +174,14 @@ static void report(void *_log, osm_epi_event_id_t event_id, void *event_data) /** ========================================================================= * Define the object symbol for loading */ + +#if OSM_EVENT_PLUGIN_INTERFACE_VER != 2 +#error OpenSM plugin interface version missmatch +#endif + osm_event_plugin_t osm_event_plugin = { - interface_version:OSM_EVENT_PLUGIN_INTERFACE_VER, - construct:construct, - destroy:destroy, + osm_version:OSM_VERSION, + create:construct, + delete:destroy, report:report }; -- 1.5.5.1.178.g1f811 From ogerlitz at voltaire.com Wed Jul 30 04:33:55 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 30 Jul 2008 14:33:55 +0300 Subject: [ofa-general][RFC] IPv6 support for RDMA CM In-Reply-To: <1217416924.32115.9.camel@linux-zn6t.site> References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: <489051A3.7030809@voltaire.com> Aleksey Senin wrote: > To perform network discovery and to symbols should be exported: > nd_table > ndisc_send_ns one quick feedback is that other then exporting nd_tbl, it is not used in this patch Or. > diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c > index beb48e3..014edd6 100644 > --- a/net/ipv6/ndisc.c > +++ b/net/ipv6/ndisc.c > @@ -156,6 +156,7 @@ struct neigh_table nd_tbl = { > .gc_thresh2 = 512, > .gc_thresh3 = 1024, > }; > +EXPORT_SYMBOL(nd_tbl); From Kent at asconex.de Wed Jul 30 04:35:19 2008 From: Kent at asconex.de (Kent Moreno) Date: Wed, 30 Jul 2008 14:35:19 +0300 Subject: [ofa-general] for Blake Message-ID: <2f4801c8f238$587b11d0$c0a80142@Kent> All the ways to long increases of manhood. Download video! http://duplika.cl/clk/new.php -------------- next part -------------- An HTML attachment was scrubbed... URL: From alekseys at voltaire.com Wed Jul 30 04:46:39 2008 From: alekseys at voltaire.com (Aleksey Senin) Date: Wed, 30 Jul 2008 14:46:39 +0300 Subject: [ofa-general][RFC] IPv6 support for RDMA CM In-Reply-To: <489051A3.7030809@voltaire.com> References: <1217416924.32115.9.camel@linux-zn6t.site> <489051A3.7030809@voltaire.com> Message-ID: <1217418399.16013.4.camel@linux-zn6t.site> This structure referenced as external in net/ndisc.h header file, that included by addr.c On Wed, 2008-07-30 at 14:33 +0300, Or Gerlitz wrote: > Aleksey Senin wrote: > > To perform network discovery and to symbols should be exported: > > nd_table > > ndisc_send_ns > one quick feedback is that other then exporting nd_tbl, it is not used > in this patch > > Or. > > > diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c > > index beb48e3..014edd6 100644 > > --- a/net/ipv6/ndisc.c > > +++ b/net/ipv6/ndisc.c > > @@ -156,6 +156,7 @@ struct neigh_table nd_tbl = { > > .gc_thresh2 = 512, > > .gc_thresh3 = 1024, > > }; > > +EXPORT_SYMBOL(nd_tbl); > > From HNGUYEN at de.ibm.com Wed Jul 30 06:05:53 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 30 Jul 2008 15:05:53 +0200 Subject: [ofa-general] Re: [PATCH] - [resend] Corrects a race in ipoib_cm_post_receive_nonsrq() In-Reply-To: Message-ID: Hi, Sorry for this very late finding as I was "blind" at the first review some weeks ago. > +static void ipoib_cm_init_rx_wr(struct net_device *dev, > + struct ib_recv_wr *wr, > + struct ib_sge *sge) > +{ > + struct ipoib_dev_priv *priv = netdev_priv(dev); > + int i; > + > + for (i = 0; i < priv->cm.num_frags; ++i) > + sge[i].lkey = priv->mr->lkey; > + > + sge[0].length = IPOIB_CM_HEAD_SIZE; > + for (i = 1; i < priv->cm.num_frags; ++i) > + sge[i].length = PAGE_SIZE; > + > + wr->next = NULL; > + wr->sg_list = priv->cm.rx_sge; This should actually be set to sge as provided by the caller. Will send a patch to fix this for 2.6.27 soon. > + wr->num_sge = priv->cm.num_frags; > +} Thanks Nam From jlentini at netapp.com Wed Jul 30 06:27:31 2008 From: jlentini at netapp.com (James Lentini) Date: Wed, 30 Jul 2008 09:27:31 -0400 (EDT) Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: <488F7357.9090004@opengridcomputing.com> References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> <469958e00807291020q33ff5b34j847066640283dff9@mail.gmail.com> <488F7357.9090004@opengridcomputing.com> Message-ID: On Tue, 29 Jul 2008, Steve Wise wrote: > Caitlin Bestler wrote: > > On Tue, Jul 29, 2008 at 9:47 AM, Felix Marti wrote: > > > > > > > Roland, AFAIK, even an unsignaled WR generates a CQE when an error > > > occurs (and then all subsequent WRs complete in error as well). Doesn't > > > that imply that the CQ must be sized assuming that _every_ WR can > > > generate a CQE? > > > > > > > > > > Yes, that is exactly what application writers should assume. > > > > They should also assume that spurious completion interrupts are possible > > (although obviously they should be minimized) and therefore the application > > should always be ready be woken up early and find nothing to do. > > > > More complete descriptions that could be used as guidelines can be found > > in the DAPL manuals. > > > > About the only thing I think that can be done about this specific quirk is > > to document it so that application developers are aware of it, and know that > > it is a model-specific limitation and do not think it is a limitation of > > RDMA > > or iWARP. If their application does not need the notification it should > > keep requesting unsignaled completion. > > > Well if applications should handle infrequent spurious notifcations > where no cqe is available, then I can indeed add code in my provider > poll to silently throw away unsignaled read completions. Did you consider returning an error from ib_post_send to indicate that an unsignalled RDMA read is not supported? james From HNGUYEN at de.ibm.com Wed Jul 30 06:42:57 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 30 Jul 2008 15:42:57 +0200 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: Message-ID: Hi Roland! > > I would say so since I didn't see the QP got destroyed. You meant > > susequents WQEs need to return flush error completion status throught > > CQ? Current ehca code level doesn't have this support. It will be fixed > > later. > > That seems like the real issue. If ehca does not generate flush error > completions for pending work requests when a QP is transitioned to the > error state, then that is a major problem that is going to break cleanup > for nearly everything. Generating flush error completions on transition > to error is a fundamental part of the IB verbs. > > So I would prefer not to work around it in IPoIB but rather fix this > basic problem in ehca. Right. We're working on a patch for ehca addressing this (flush error cqes) and will post it here for review, probably in 2-3 weeks. Thx Nam From rdreier at cisco.com Wed Jul 30 07:22:51 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 07:22:51 -0700 Subject: [ofa-general] Re: [PATCH] - [resend] Corrects a race in ipoib_cm_post_receive_nonsrq() In-Reply-To: (Hoang-Nam Nguyen's message of "Wed, 30 Jul 2008 15:05:53 +0200") References: Message-ID: thanks, I'll queue up this patch: commit e08198169ec5facb3d85bb455efa44a2f8327842 Author: Roland Dreier Date: Wed Jul 30 07:21:46 2008 -0700 IPoIB/cm: Set correct SG list in ipoib_cm_init_rx_wr() wr->sg_list should be set to the sge pointer passed in, not priv->cm.rx_sge. Reported-by: Hoang-Nam Nguyen Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 0f2d304..7ebc400 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -337,7 +337,7 @@ static void ipoib_cm_init_rx_wr(struct net_device *dev, sge[i].length = PAGE_SIZE; wr->next = NULL; - wr->sg_list = priv->cm.rx_sge; + wr->sg_list = sge; wr->num_sge = priv->cm.num_frags; } From alekseys at voltaire.com Wed Jul 30 07:44:38 2008 From: alekseys at voltaire.com (Aleksey Senin) Date: Wed, 30 Jul 2008 17:44:38 +0300 Subject: [ofa-general] [RFC PATCH] IPv6 support for examples/rping.c Message-ID: <1217429078.1230.3.camel@linux-zn6t.site> Here is patch that should be applied to rping.c in order to check RDMA CM support in the kernel. >From 40acd529ae583f0e7ec88a070ad55cc9a3c430d4 Mon Sep 17 00:00:00 2001 From: Aleksey Senin Date: Wed, 30 Jul 2008 17:30:49 +0300 Subject: [PATCH] Initial support for IPv6 Signed-off-by: Aleksey Senin --- examples/rping.c | 81 ++++++++++++++++++++++++++++++++++++++++++++--------- 1 files changed, 67 insertions(+), 14 deletions(-) diff --git a/examples/rping.c b/examples/rping.c index 983ce1c..cbfcc1c 100644 --- a/examples/rping.c +++ b/examples/rping.c @@ -145,7 +145,9 @@ struct rping_cb { uint16_t port; /* dst port in NBO */ uint32_t addr; /* dst addr in NBO */ + uint32_t addr_pad[3]; char *addr_str; /* dst addr string */ + char *laddr_str; int verbose; /* verbose logging */ int count; /* ping count */ int size; /* ping data size */ @@ -727,17 +729,36 @@ static int rping_test_server(struct rping_cb *cb) return ret; } +struct rping_addr{ + struct sockaddr addr; + unsigned char pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; +}; + + static int rping_bind_server(struct rping_cb *cb) { - struct sockaddr_in sin; + struct rping_addr sin; int ret; - memset(&sin, 0, sizeof(sin)); - sin.sin_family = AF_INET; - sin.sin_addr.s_addr = cb->addr; - sin.sin_port = cb->port; - ret = rdma_bind_addr(cb->cm_id, (struct sockaddr *) &sin); + if ( cb->addr_str && strchr(cb->addr_str, ':') ){ + DEBUG_LOG("Got IPv6 address: %s\n", cb->addr_str); + struct sockaddr_in6 *addr = ( struct sockaddr_in6*)&sin.addr; + sin.addr.sa_family = AF_INET6; + addr->sin6_port = cb->port; + inet_pton(AF_INET6, cb->addr_str, addr->sin6_addr.s6_addr); + /* &sin.r6_addr.sin6_addr.s6_addr); */ + } else { + struct sockaddr_in *addr = (struct sockaddr_in*)&sin.addr; + DEBUG_LOG("Got IPv4 address: %s\n", cb->addr_str); + sin.addr.sa_family = AF_INET; + addr->sin_port = cb->port; + addr->sin_addr.s_addr = cb->addr; + /*sin.r_addr.sin_port = cb->port; + sin.r_addr.sin_addr.s_addr = cb->addr; */ + } + + ret = rdma_bind_addr(cb->cm_id, &sin.addr); if (ret) { fprintf(stderr, "rdma_bind_addr error %d\n", ret); return ret; @@ -997,16 +1018,43 @@ static int rping_connect_client(struct rping_cb *cb) static int rping_bind_client(struct rping_cb *cb) { - struct sockaddr_in sin; + struct rping_addr sin; + struct rping_addr saddr; + struct sockaddr *paddr; int ret; memset(&sin, 0, sizeof(sin)); - sin.sin_family = AF_INET; - sin.sin_addr.s_addr = cb->addr; - sin.sin_port = cb->port; - ret = rdma_resolve_addr(cb->cm_id, NULL, (struct sockaddr *) &sin, - 2000); + if(cb->addr_str && strchr(cb->addr_str, ':') ) { + struct sockaddr_in6 *addr = (struct sockaddr_in6*)&sin.addr; + sin.addr.sa_family = AF_INET6; + addr->sin6_port = cb->port; + inet_pton(AF_INET6, cb->addr_str, addr->sin6_addr.s6_addr); + } else { + struct sockaddr_in *addr = (struct sockaddr_in*)&sin.addr; + sin.addr.sa_family = AF_INET; + addr->sin_addr.s_addr = cb->addr; + addr->sin_port = cb->port; + } + if ( sin.addr.sa_family == AF_INET6 ) { + struct sockaddr_in6 *addr = (struct sockaddr_in6*)&saddr.addr; + saddr.addr.sa_family = AF_INET6; + addr->sin6_port = 7174; + if(cb->laddr_str) { + DEBUG_LOG("Using specified address: %s\n", cb->laddr_str); + inet_pton(AF_INET6, cb->laddr_str, addr->sin6_addr.s6_addr); + paddr = &saddr.addr; + } + else { + DEBUG_LOG("You have to specify local IPv6 address in order to use this programm with IPv6 support"); + return -1; + } + }else { + paddr = NULL; + } + + ret = rdma_resolve_addr(cb->cm_id, paddr, &sin.addr, 2000); + if (ret) { fprintf(stderr, "rdma_resolve_addr error %d\n", ret); return ret; @@ -1084,6 +1132,7 @@ static void usage(char *name) printf("\t-a addr\t\taddress\n"); printf("\t-p port\t\tport\n"); printf("\t-P\t\tpersistent server mode allowing multiple connections\n"); + printf("\t-I\t\tlocal IPv6 source address\n"); } int main(int argc, char *argv[]) @@ -1105,11 +1154,12 @@ int main(int argc, char *argv[]) sem_init(&cb->sem, 0, 0); opterr = 0; - while ((op=getopt(argc, argv, "a:Pp:C:S:t:scvVd")) != -1) { + while ((op=getopt(argc, argv, "a:Pp:C:S:t:scvVdI:")) != -1) { switch (op) { case 'a': cb->addr_str = optarg; - cb->addr = inet_addr(optarg); + strchr(optarg, ':')?inet_pton(AF_INET6, optarg, &cb->addr):\ + inet_pton(AF_INET, optarg, &cb->addr); DEBUG_LOG("ipaddr (%s)\n", optarg); break; case 'P': @@ -1158,6 +1208,9 @@ int main(int argc, char *argv[]) case 'd': debug++; break; + case 'I': + cb->laddr_str = optarg; + break; default: usage("rping"); ret = EINVAL; -- 1.5.6.dirty From HNGUYEN at de.ibm.com Wed Jul 30 08:03:42 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 30 Jul 2008 17:03:42 +0200 Subject: [ofa-general] Re: [PATCH] - [resend] Corrects a race in ipoib_cm_post_receive_nonsrq() In-Reply-To: Message-ID: Thanks, you're much faster than me:-) Nam Roland Dreier wrote on 30.07.2008 16:22:51: > thanks, I'll queue up this patch: > > commit e08198169ec5facb3d85bb455efa44a2f8327842 > Author: Roland Dreier > Date: Wed Jul 30 07:21:46 2008 -0700 > > IPoIB/cm: Set correct SG list in ipoib_cm_init_rx_wr() > > wr->sg_list should be set to the sge pointer passed in, not > priv->cm.rx_sge. > > Reported-by: Hoang-Nam Nguyen > Signed-off-by: Roland Dreier > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > index 0f2d304..7ebc400 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > @@ -337,7 +337,7 @@ static void ipoib_cm_init_rx_wr(struct net_device *dev, > sge[i].length = PAGE_SIZE; > > wr->next = NULL; > - wr->sg_list = priv->cm.rx_sge; > + wr->sg_list = sge; > wr->num_sge = priv->cm.num_frags; > } > From hal.rosenstock at gmail.com Wed Jul 30 08:13:26 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 30 Jul 2008 11:13:26 -0400 Subject: [ofa-general] [PATCH] opensm/osm_ucast_lash.c: A couple more conversions to (unsigned) decimal lid format Message-ID: Sasha, Please see attached file. -- Hal -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-lash1 Type: application/octet-stream Size: 1176 bytes Desc: not available URL: From pmorreale at novell.com Wed Jul 30 08:23:25 2008 From: pmorreale at novell.com (Peter W. Morreale) Date: Wed, 30 Jul 2008 09:23:25 -0600 Subject: [ofa-general] ibv_alloc_pd seg fault.... Message-ID: <1217431405.23124.25.camel@hermosa.site> Hello all, I'm a noob attempting to create a client/server using RDMA as a transport and running into what is clearly a coding issue on my part. Right now, the client-side of my app is seg faulting in ibv_alloc_pd(), and I'm a little confused as to why. I'm using OFED 1.3 on SLERT SP2 (SUSE Linux Enterprise Real Time). I'm using Roland's client/server code as an example. This code sends two integers from the client, adds them on the server side, and sends back the result to the client. This code works on my systems. Right now I'm to the point of sending a simple test message from the client to the server. The app is constructed such that each side is both a client and server to the other side. During initialization, I'm creating a pthread to act as the server to the other side. The app creates two channels, one for use by the client-side, and one for use by the server-side. I assume this is required, correct? The client-side then creates a connection to the server via: rdma_create_event_channel() rdma_create_id() rdma_resolve_route() rdma_get_cm_event() test for RDMA_CM_EVENT_ADDR_RESOLVED rdma_ack_event() rdma_resolve_route() rmda_get_cm_event() test for RDMA_CM_EVENT_ROUTE_RESOLVED rdma_ack_event() ibv_alloc_pd() <-- seg faults. The cma_id is located on the stack, and properly defined down to this point, but that is not an issue correct? Looking at things in gdb, I see that cma_id is initialized with what appear to be a series of callbacks for various operations, as well as some other data. Any ideas as to what I may be doing wrong? Thanks in advance. Best, -PWM From hal.rosenstock at gmail.com Wed Jul 30 08:28:23 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 30 Jul 2008 11:28:23 -0400 Subject: [ofa-general] Re: opensm/osm_sa.c: GID format In-Reply-To: <20080729232930.GM5487@sashak.voltaire.com> References: <20080729232930.GM5487@sashak.voltaire.com> Message-ID: On Tue, Jul 29, 2008 at 7:29 PM, Sasha Khapyorsky wrote: > On 10:07 Mon 28 Jul , Hal Rosenstock wrote: >> Should the GID format in opensm/osm_sa.c also be updated to use >> inet_ntop/inet_pton rather than PRIx64 ? > > I have nothing against this. > >> If so, should it also support >> the old format for input so old files can still be imported ? > > Seems like an overkill for me. So there is no need for older dump files to work with newer OpenSM ? If not, that makes this a little simpler. -- Hal > Sasha > From or.gerlitz at gmail.com Tue Jul 29 13:44:33 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 29 Jul 2008 23:44:33 +0300 Subject: ***SPAM*** Re: [ofa-general] [PATCH 0/12 V2] XRC patch series, for 2.6.27 In-Reply-To: <200807101851.11045.jackm@dev.mellanox.co.il> References: <200807101851.11045.jackm@dev.mellanox.co.il> Message-ID: <15ddcffd0807291344t6fe61701x78fc5b9127bf7a3f@mail.gmail.com> On Thu, Jul 10, 2008 at 6:51 PM, Jack Morgenstein wrote: > This is the updated patch series, incorporating all of the fixes and > changes based on Roland's and Ron Livne's feedback. This series is ready for > Ron's proposed changes to be integrated. Roland, I just wanted to check with you if regardless of the fact that XRC didn't make it to 2.6.27-rc1, the review / merge process of XRC and Ron's patches for create-qp-expanded keeps going from your side till the patches gets into the for-2.6.28 etc branch of yours. Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vgurbboqlqu at lebanonwire.every1.net Wed Jul 30 10:12:02 2008 From: vgurbboqlqu at lebanonwire.every1.net (kelli) Date: Wed, 30 Jul 2008 09:12:02 -0800 Subject: [ofa-general] it`s kelli Message-ID: <1-3451930-qU3BznfbfA45IYgiELwsCFk@atr.interaxel2.com> hello, I am pretty russian girl, bored tonight. would you like to chat with me and see my pics? if so then email me at ekelli006 at rdplanet.com From rdreier at cisco.com Wed Jul 30 09:48:53 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 09:48:53 -0700 Subject: [ofa-general][RFC] IPv6 support for RDMA CM In-Reply-To: <1217416924.32115.9.camel@linux-zn6t.site> (Aleksey Senin's message of "Wed, 30 Jul 2008 14:22:04 +0300") References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: haven't looked in detail, but a couple quick comments: > struct sockaddr src_addr; > + u8 src_pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; Rather than doing this (which is rather fragile in addition to being ugly), just use struct sockaddr_storage. Yes the existing CMA code should get the same treatment. > --- a/net/ipv6/ndisc.c > +++ b/net/ipv6/ndisc.c > @@ -156,6 +156,7 @@ struct neigh_table nd_tbl = { > .gc_thresh2 = 512, > .gc_thresh3 = 1024, > }; > +EXPORT_SYMBOL(nd_tbl); Rather than exporting nd_tbl I think it would make more sense to uninline ndisc_get_neigh() and export that. I suggest starting a discussion about this on netdev at vger.kernel.org -- explain the rationale of wanting to do IPv6 neighbour discovery from a module and post this patch including the part that exports the needed core symbols. - R. From rdreier at cisco.com Wed Jul 30 09:57:34 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 09:57:34 -0700 Subject: [ofa-general] [PATCH/RFC] RDMA/cma: Remove padding arrays by using struct sockaddr_storage In-Reply-To: (Roland Dreier's message of "Wed, 30 Jul 2008 09:48:53 -0700") References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: There are a few places that the RDMA CM code handles IPv6 by doing struct sockaddr addr; u8 pad[sizeof(struct sockaddr_in6) - sizeof(struct sockaddr)]; This is fragile and ugly; handle this in a better way with just struct sockaddr_storage addr; Signed-off-by: Roland Dreier --- Any objections to merging the cleanup below? drivers/infiniband/core/cma.c | 4 +--- drivers/infiniband/core/ucma.c | 10 ++++------ 2 files changed, 5 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index e980ff3..a16510b 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -155,9 +155,7 @@ struct cma_multicast { } multicast; struct list_head list; void *context; - struct sockaddr addr; - u8 pad[sizeof(struct sockaddr_in6) - - sizeof(struct sockaddr)]; + struct sockaddr_storage addr; }; struct cma_work { diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c index b41dd26..d5a825f 100644 --- a/drivers/infiniband/core/ucma.c +++ b/drivers/infiniband/core/ucma.c @@ -81,9 +81,7 @@ struct ucma_multicast { u64 uid; struct list_head list; - struct sockaddr addr; - u8 pad[sizeof(struct sockaddr_in6) - - sizeof(struct sockaddr)]; + struct sockaddr_storage addr; }; struct ucma_event { @@ -913,7 +911,7 @@ static ssize_t ucma_join_multicast(struct ucma_file *file, mc->uid = cmd.uid; memcpy(&mc->addr, &cmd.addr, sizeof cmd.addr); - ret = rdma_join_multicast(ctx->cm_id, &mc->addr, mc); + ret = rdma_join_multicast(ctx->cm_id, (struct sockaddr *) &mc->addr, mc); if (ret) goto err2; @@ -929,7 +927,7 @@ static ssize_t ucma_join_multicast(struct ucma_file *file, return 0; err3: - rdma_leave_multicast(ctx->cm_id, &mc->addr); + rdma_leave_multicast(ctx->cm_id, (struct sockaddr *) &mc->addr); ucma_cleanup_mc_events(mc); err2: mutex_lock(&mut); @@ -975,7 +973,7 @@ static ssize_t ucma_leave_multicast(struct ucma_file *file, goto out; } - rdma_leave_multicast(mc->ctx->cm_id, &mc->addr); + rdma_leave_multicast(mc->ctx->cm_id, (struct sockaddr *) &mc->addr); mutex_lock(&mc->ctx->file->mut); ucma_cleanup_mc_events(mc); list_del(&mc->list); From rdreier at cisco.com Wed Jul 30 10:03:18 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 10:03:18 -0700 Subject: [ofa-general][RFC] IPv6 support for RDMA CM In-Reply-To: (Roland Dreier's message of "Wed, 30 Jul 2008 09:48:53 -0700") References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: > Rather than exporting nd_tbl I think it would make more sense to > uninline ndisc_get_neigh() and export that. FWIW, the patch below shrinks the kernel code somewhat: add/remove: 1/0 grow/shrink: 1/5 up/down: 81/-217 (-136) function old new delta ndisc_get_neigh - 76 +76 ipv6_add_addr 644 649 +5 ip6_mc_msfilter 665 663 -2 tcp_sendmsg 2558 2550 -8 icmp6_dst_alloc 332 284 -48 rt6_alloc_cow 266 201 -65 addrconf_dst_alloc 433 339 -94 and I don't think ndisc_get_neigh() is on any fast path. So this is probably a good approach. diff --git a/include/net/ndisc.h b/include/net/ndisc.h index a01b7c4..e39e5c0 100644 --- a/include/net/ndisc.h +++ b/include/net/ndisc.h @@ -138,15 +138,7 @@ int ndisc_ifinfo_sysctl_strategy(ctl_table *ctl, int __user *name, extern void inet6_ifinfo_notify(int event, struct inet6_dev *idev); -static inline struct neighbour * ndisc_get_neigh(struct net_device *dev, const struct in6_addr *addr) -{ - - if (dev) - return __neigh_lookup(&nd_tbl, addr, dev, 1); - - return NULL; -} - +struct neighbour *ndisc_get_neigh(struct net_device *dev, const struct in6_addr *addr); #endif /* __KERNEL__ */ diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index beb48e3..1789b4c 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -1593,6 +1593,15 @@ void ndisc_send_redirect(struct sk_buff *skb, struct neighbour *neigh, in6_dev_put(idev); } +struct neighbour *ndisc_get_neigh(struct net_device *dev, const struct in6_addr *addr) +{ + + if (dev) + return __neigh_lookup(&nd_tbl, addr, dev, 1); + + return NULL; +} + static void pndisc_redo(struct sk_buff *skb) { ndisc_recv_ns(skb); From Caitlin.Bestler at neterion.com Wed Jul 30 10:10:12 2008 From: Caitlin.Bestler at neterion.com (Caitlin Bestler) Date: Wed, 30 Jul 2008 13:10:12 -0400 Subject: [ofa-general] Non-Signaled RDMA Read (cxgb3) In-Reply-To: References: <488F4042.4010108@opengridcomputing.com> <488F461B.5060602@opengridcomputing.com> <8A71B368A89016469F72CD08050AD334032EC49A@maui.asicdesigners.com> <469958e00807291020q33ff5b34j847066640283dff9@mail.gmail.com> <488F7357.9090004@opengridcomputing.com> Message-ID: <78C9135A3D2ECE4B8162EBDCE82CAD7703FA4947@nekter> James Lentini wrote: > > Well if applications should handle infrequent spurious notifcations > > where no cqe is available, then I can indeed add code in my provider > > poll to silently throw away unsignaled read completions. > > Did you consider returning an error from ib_post_send to indicate > that an unsignalled RDMA read is not supported? > > james Suppressing notifications is a performance optimization, not a correctness issue. That is, while it is certainly desirable for a device to provide no more notifications than required, the application ultimately has to be able to deal with spurious notifications. Implementations ultimately have to choose between very costly synchronization, too few notifications or too many notifications. Clearly, slightly too many is far preferable than slightly too few. And how much effort is made to keep that gap small is a question best left to the marketplace. Given that, I don't see why the inability to honor an optimization request should be flagged as an "error". From jlgreath at us.ibm.com Wed Jul 30 11:43:59 2008 From: jlgreath at us.ibm.com (Joseph L Greathouse) Date: Wed, 30 Jul 2008 13:43:59 -0500 Subject: [ofa-general] Receiving 24-byte remnant header on bad umad_recv? Message-ID: Hi there, I am building a program that sometimes sends MADs from device A to device B, and device B should (by design) silently ignore these packets. Device A is using OFED 1.3.1's libibumad-1.1.7, and I am sending these MADs using a regular umad_send() method. When B responds, the subsequent umad_recv() works just fine. However, when B is supposed to silently ignores the MAD, device A's subsequent umad_recv() receives a strange packet rather than timing out. It receives a 24-byte packet that looks like a MAD Header. All of its fields except for TID are the same as the original send's MAD Header. I have verified that, at the very least, device B is *not* sending out this 24 byte packet. In fact, this 24-byte malformed packet appears even when doing a similar test from device A to device C (where device C is from another manufacturer). Even using umad_poll() to wait for an actual response results in this misformed packet. umad_poll() returns with 0, and the next umad_recv() picks up the 24-byte remnant heaer. Is this the expected case in OFED when a MAD send fails to receive a response? Does receiving a 24-byte packet using umad_recv() guarantee that the response was never received, or could this arise in other situations as well? Any help you could offer would be appreciated. -Joe Greathouse From pmorreale at novell.com Wed Jul 30 11:50:01 2008 From: pmorreale at novell.com (Peter W. Morreale) Date: Wed, 30 Jul 2008 12:50:01 -0600 Subject: [ofa-general] ibv_alloc_pd seg fault.... In-Reply-To: <1217431405.23124.25.camel@hermosa.site> References: <1217431405.23124.25.camel@hermosa.site> Message-ID: <1217443801.23124.28.camel@hermosa.site> I found my problem. Apparently an issue with the shared lib I was creating. Thx, -PWM On Wed, 2008-07-30 at 09:23 -0600, Peter W. Morreale wrote: > Hello all, > > I'm a noob attempting to create a client/server using RDMA as a > transport and running into what is clearly a coding issue on my part. > > Right now, the client-side of my app is seg faulting in ibv_alloc_pd(), > and I'm a little confused as to why. > > I'm using OFED 1.3 on SLERT SP2 (SUSE Linux Enterprise Real Time). I'm > using Roland's client/server code as an example. This code sends two > integers from the client, adds them on the server side, and sends back > the result to the client. This code works on my systems. > > Right now I'm to the point of sending a simple test message from the > client to the server. > > The app is constructed such that each side is both a client and server > to the other side. During initialization, I'm creating a pthread to act > as the server to the other side. > > The app creates two channels, one for use by the client-side, and one > for use by the server-side. I assume this is required, correct? > > The client-side then creates a connection to the server via: > > rdma_create_event_channel() > rdma_create_id() > rdma_resolve_route() > rdma_get_cm_event() > test for RDMA_CM_EVENT_ADDR_RESOLVED > rdma_ack_event() > rdma_resolve_route() > rmda_get_cm_event() > test for RDMA_CM_EVENT_ROUTE_RESOLVED > rdma_ack_event() > ibv_alloc_pd() <-- seg faults. > > The cma_id is located on the stack, and properly defined down to this > point, but that is not an issue correct? > > Looking at things in gdb, I see that cma_id is initialized with what > appear to be a series of callbacks for various operations, as well as > some other data. > > Any ideas as to what I may be doing wrong? > > Thanks in advance. > > Best, > -PWM > > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From rdreier at cisco.com Wed Jul 30 12:02:35 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 12:02:35 -0700 Subject: [ofa-general] Receiving 24-byte remnant header on bad umad_recv? In-Reply-To: (Joseph L. Greathouse's message of "Wed, 30 Jul 2008 13:43:59 -0500") References: Message-ID: > I am building a program that sometimes sends MADs from device A to device > B, and device B should (by design) silently ignore these packets. > > Device A is using OFED 1.3.1's libibumad-1.1.7, and I am sending these MADs > using a regular umad_send() method. When B responds, the subsequent > umad_recv() works just fine. However, when B is supposed to silently > ignores the MAD, device A's subsequent umad_recv() receives a strange > packet rather than timing out. It receives a 24-byte packet that looks > like a MAD Header. All of its fields except for TID are the same as the > original send's MAD Header. This is expected... the small packet is showing that device A's request is timing out (and you should see the status field in the packet showing a timeout error -- I forget the exact details but you can look at the user_mad.c source in the kernel to see what happens in this case). From jlgreath at us.ibm.com Wed Jul 30 12:56:49 2008 From: jlgreath at us.ibm.com (Joseph L Greathouse) Date: Wed, 30 Jul 2008 14:56:49 -0500 Subject: [ofa-general] Receiving 24-byte remnant header on bad umad_recv? In-Reply-To: Message-ID: >> I am building a program that sometimes sends MADs from device A to >> device B, and device B should (by design) silently ignore these >> packets., >> >> Device A is using OFED 1.3.1's libibumad-1.1.7, and I am sending >> these MADs using a regular umad_send() method. When B responds, >> the subsequent umad_recv() works just fine. However, when B is >> supposed to silently ignores the MAD, device A's subsequent >> umad_recv() receives a strange packet rather than timing out. >> It receives a 24-byte packet that looks like a MAD Header. All of >> its fields except for TID are the same as the original send's MAD >> Header. >> >This is expected... the small packet is showing that device A's >request is timing out (and you should see the status field in the >packet showing a timeout error -- I forget the exact details but you >can look at the user_mad.c source in the kernel to see what happens >in this case). Aha! That makes a lot more sense now. Thank you. -Joe Greathouse From haberci at haftasonuevi.com Wed Jul 30 16:00:34 2008 From: haberci at haftasonuevi.com (Haftasonu Evi) Date: Thu, 31 Jul 2008 02:00:34 +0300 Subject: [ofa-general] =?windows-1254?q?Ankara=27da_K=FCt=FCk_Ev_Projesi?= Message-ID: <41178-22008733023034411@xx-> Ankara Çubuk Karagöl'e yaklaşık 800 metre mesafede, Yedigöllerde olduğu gibi renk cümbüşü ormanda trekking, bisiklete binme, kısa yürüyüşler yapabileceğimiz , kuşburnu, alıç ve ahlat gibi yabani meyveler toplayıp, günün bitiminde ormanın yanıbaşında huzurlu kısa tatillerimizi geçireceğimiz mütevazi kütük evimizde konaklayacağımız, organik tarım çerçevesinde hem sebze , hem de ciddi gelir getirici ceviz ağacı yetiştirmek amacıyla 60 dönüm tapulu bir arazi satın aldık. Bizlerle aynı duyguları paylaşan, şehrin beton ve yapaylığından bıkıp ormanda ki doğal ortamda kendi evinde ve kocaman bahçesinde huzur bulmak isteyen çok az sayıda doğa severe, sizlere sesleniyoruz.Projeyi yapanlarında projenin içerisinde yer aldığı , bu on evlik proje hakkında daha ayrıntılı bilgi için : http://www.haftasonuevi.com/ adresini inceleyebilirsiniz. From bill at cse.ucdavis.edu Wed Jul 30 17:08:08 2008 From: bill at cse.ucdavis.edu (Bill Broadley) Date: Wed, 30 Jul 2008 17:08:08 -0700 Subject: [ofa-general] Infinipath fix for OFED-1.3.1 In-Reply-To: References: Message-ID: <48910268.8050606@cse.ucdavis.edu> Turns out the infinipath requires openmpi to be built with --with-psm-dir, so here's the patch: # diff install.pl install.pl-dist 3013d3012 < $openmpi_comp_env .= " --with-psm-dir=/usr"; Also it's required that the qlogic supplied infinipath, infinipath-devel, and infinipath-kernel are installed before the OFED rpms which provide openib and openmpi. Thanks to the OFED folks we now have a MPI environment that supports GigE, Infinipath, GCC-4.3.1, and Pathscale compiler. Although we were not able to build OFED-4.3.1 with GCC-4.3.1, we were able to build openmpi with GCC-4.3.1. Just figured we would let folks know in case anyone else wants to get OFED working with Infinipath. From jsquyres at cisco.com Wed Jul 30 17:15:28 2008 From: jsquyres at cisco.com (Jeff Squyres) Date: Wed, 30 Jul 2008 20:15:28 -0400 Subject: [ofa-general] Infinipath fix for OFED-1.3.1 In-Reply-To: <48910268.8050606@cse.ucdavis.edu> References: <48910268.8050606@cse.ucdavis.edu> Message-ID: <81F617B2-072B-4CBD-98D7-6D9E71E7A3DB@cisco.com> On Jul 30, 2008, at 8:08 PM, Bill Broadley wrote: > Turns out the infinipath requires openmpi to be built with --with- > psm-dir, so here's the patch: > # diff install.pl install.pl-dist > 3013d3012 > < $openmpi_comp_env .= " --with-psm-dir=/usr"; Hmm; that's odd. OMPI should pick up the PSM libraries by default if they're in the linker's default path. > Also it's required that the qlogic supplied infinipath, infinipath- > devel, and infinipath-kernel are installed before the OFED rpms > which provide openib and openmpi. > > Thanks to the OFED folks we now have a MPI environment that supports > GigE, Infinipath, GCC-4.3.1, and Pathscale compiler. Although we > were not able to build OFED-4.3.1 with GCC-4.3.1, we were able to > build openmpi with GCC-4.3.1. Note that there is an outstanding issue with the Pathscale C++ compiler and the Open MPI C++ bindings: https://svn.open-mpi.org/trac/ompi/ticket/1326 It *looks* like a Pathscale bug (some C++ constructors are definitely not firing), but I don't know for sure. The ball is in Pathscale's court -- I initially reported it about 2 months ago; our last e-mail exchange about it was within the last week or two. -- Jeff Squyres Cisco Systems From weiny2 at llnl.gov Wed Jul 30 17:40:11 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 30 Jul 2008 17:40:11 -0700 Subject: [ofa-general] [PATCH] ibsim: Add a Node Description query drop error. Message-ID: <20080730174011.5d80e036.weiny2@llnl.gov> I added this to ibsim to test the patch I am sending next: [PATCH] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. Thanks, Ira >From ec95833fd08204cb67c9803b488c6ef7e9a30792 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 30 Jul 2008 16:16:11 -0700 Subject: [PATCH] ibsim: Add a Node Description query drop error. Signed-off-by: Ira K. Weiny --- ibsim/sim.h | 1 + ibsim/sim_cmd.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ ibsim/sim_mad.c | 8 ++++++ 3 files changed, 83 insertions(+), 0 deletions(-) diff --git a/ibsim/sim.h b/ibsim/sim.h index 936bb85..3cd253f 100644 --- a/ibsim/sim.h +++ b/ibsim/sim.h @@ -207,6 +207,7 @@ struct Port { Node *remotenode; int remoteport; int errrate; + int nddrop; Node *node; Portcounters portcounters; uint16_t *pkey_tbl; diff --git a/ibsim/sim_cmd.c b/ibsim/sim_cmd.c index a6aab9d..fbf2ca3 100644 --- a/ibsim/sim_cmd.c +++ b/ibsim/sim_cmd.c @@ -262,6 +262,74 @@ static int do_seterror(FILE * f, char *line) return set; } +static int do_setnddrop(FILE * f, char *line) +{ + Port *port, *e; + Node *node; + char *s = line; + char *nodeid = 0, name[NAMELEN], *sp, *orig = 0; + int portnum = -1; // def - all ports + int numports, set = 0, rate = 0; + + if (strsep(&s, "\"")) + orig = strsep(&s, "\""); + + if (!s) { + fprintf(f, "# unlink: bad parameter in \"%s\"\n", line); + return -1; + } + + nodeid = expand_name(orig, name, &sp); + if (!sp && *s == '[') + sp = s + 1; + + if (!(node = find_node(nodeid))) { + fprintf(f, "# nodeid \"%s\" (%s) not found\n", orig, nodeid); + return -1; + } + + if (sp) { + portnum = strtoul(sp, 0, 0); + if (portnum < 1 || portnum > node->numports) { + fprintf(f, "# bad port number %d at nodeid \"%s\"\n", + portnum, nodeid); + return -1; + } + } + + numports = node->numports; + + if (node->type == SWITCH_NODE) + numports++; // To make the for-loop below run up to last port + else + portnum--; + + if (portnum >= 0) { + port = ports + node->portsbase + portnum; + if (port->nddrop) + port->nddrop = 0; + else + port->nddrop = 1; + DEBUG("nddrop is %d", port->nddrop); + printf("Setting NDDrop %s:%d\n", port->node->nodedesc, port->portnum); + return 1; + } + + for (port = ports + node->portsbase, e = port + numports; port < e; + port++) { + port->errrate = rate; + if (port->nddrop) + port->nddrop = 0; + else + port->nddrop = 1; + DEBUG("nddrop is %d", port->nddrop); + printf("Setting NDDrop %s:%d\n", port->node->nodedesc, port->portnum); + } + + return set; + +} + static int do_unlink(FILE * f, char *line, int clear) { Port *port, *e; @@ -710,6 +778,8 @@ static int dump_help(FILE * f) fprintf(f, "\tError \"nodeid\"[port] : set error rate for port/node\n"); fprintf(f, + "\tNDError \"nodeid\"[port]: Toggle the dropping of Node Description packets for the port/node\n"); + fprintf(f, "\tBaselid \"nodeid\"[port] [lmc] : change port's lid (lmc)\n"); fprintf(f, "\tVerbose [newlevel] - show/set simulator verbosity\n"); fprintf(f, "\t\t\t0 - silent\n"); @@ -773,6 +843,10 @@ int do_cmd(char *buf, FILE *f) case 'E': r = do_seterror(f, line); break; + case 'n': + case 'N': + r = do_setnddrop(f, line); + break; case 'c': case 'C': r = do_unlink(f, line, 1); diff --git a/ibsim/sim_mad.c b/ibsim/sim_mad.c index b8ce2ab..ac3321e 100644 --- a/ibsim/sim_mad.c +++ b/ibsim/sim_mad.c @@ -1188,6 +1188,14 @@ int process_packet(Client * cl, void *p, int size, Client ** dcl) return sizeof(*r); // forward only } + /* check for node desc drop. */ + if ((rpc.mgtclass == IB_SMI_CLASS || rpc.mgtclass == IB_SMI_DIRECT_CLASS) && + rpc.attr.id == IB_ATTR_NODE_DESC && + port->nddrop) { + VERB("drop pkt due to nddrop flag %d", port->nddrop); + goto _dropped; + } + if (port->errrate && (random() % 100) < port->errrate) { VERB("drop pkt due error rate %d", port->errrate); goto _dropped; -- 1.5.4.5 From weiny2 at llnl.gov Wed Jul 30 17:40:17 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 30 Jul 2008 17:40:17 -0700 Subject: [ofa-general] [PATCH] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. Message-ID: <20080730174017.488f81c1.weiny2@llnl.gov> >From d2e52a8b9de02521b01de3414562f45e476cafbf Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 30 Jul 2008 17:28:30 -0700 Subject: [PATCH] Add a Node Description check on light sweep to ensure that the ND has been found for each node. This case covers the condition where a ND message is dropped/lost for some reason and OpenSM is left with a valid configured node which is not named correctly. This is not the same as a node which has changed it's Node Descriptioin. In this case the node needs to send a trap. Signed-off-by: Ira K. Weiny --- opensm/opensm/osm_state_mgr.c | 54 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 54 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index b599582..15124c9 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -506,6 +506,54 @@ Exit: } /********************************************************************** + During a light sweep check each node to see if the node descriptor is valid + if not issue a ND query. +**********************************************************************/ +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, + IN void *context) +{ + osm_physp_t *p_physp = NULL; + osm_node_t *const p_node = (osm_node_t *) p_object; + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t mad_context; + osm_sm_t *sm = (osm_sm_t *)context; + + OSM_LOG_ENTER(sm->p_log); + + CL_ASSERT(p_node); + + if (p_node->print_desc && strcmp(p_node->print_desc, "")) + /* if ND is valid, do nothing */ + goto exit; + + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "__osm_state_mgr_get_node_desc: " + "Unknown node description \"%s\" for node 0x%016" PRIx64 + ". Reissuing ND query\n", + p_node->print_desc ? p_node->print_desc : "", + cl_ntoh64(osm_node_get_node_guid (p_node))); + + /* get a physp to request from. */ + p_physp = osm_node_get_any_physp_ptr(p_node); + + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); + + status = osm_req_get(sm, + osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_NODE_DESC, + 0, CL_DISP_MSGID_NONE, &mad_context); + if (status != IB_SUCCESS) + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_get_node_desc: ERR 0D03: " + "Failure initiating NodeDescription request (%s)\n", + ib_get_err_str(status)); + +exit: + OSM_LOG_EXIT(sm->p_log); +} + + +/********************************************************************** Initiates a lightweight sweep of the subnet. Used during normal sweeps after the subnet is up. **********************************************************************/ @@ -514,6 +562,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) ib_api_status_t status = IB_SUCCESS; osm_bind_handle_t h_bind; cl_qmap_t *p_sw_tbl; + cl_qmap_t *p_node_tbl; cl_map_item_t *p_next; osm_node_t *p_node; osm_physp_t *p_physp; @@ -522,6 +571,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) OSM_LOG_ENTER(sm->p_log); p_sw_tbl = &sm->p_subn->sw_guid_tbl; + p_node_tbl = &sm->p_subn->node_guid_tbl; /* * First, get the bind handle. @@ -540,6 +590,10 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) cl_qmap_apply_func(p_sw_tbl, __osm_state_mgr_get_sw_info, sm); CL_PLOCK_RELEASE(sm->p_lock); + CL_PLOCK_ACQUIRE(sm->p_lock); + cl_qmap_apply_func(p_node_tbl, __osm_state_mgr_get_node_desc, sm); + CL_PLOCK_RELEASE(sm->p_lock); + /* now scan the list of physical ports that were not down but have no remote port */ CL_PLOCK_ACQUIRE(sm->p_lock); p_next = cl_qmap_head(&sm->p_subn->node_guid_tbl); -- 1.5.4.5 From rdreier at cisco.com Wed Jul 30 20:23:51 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 20:23:51 -0700 Subject: [ofa-general] XRC merging status Message-ID: I've been working on merging the XRC patch set. I've made a little progress -- I resplit the core patches so I could understand them (and fixed a few bugs in the process). The kernel receive QP and mlx4 changes still need review, and I'm hoping that the kernel receive QP stuff can be made simpler (at least the mlx4 part). Anyway, here's what I have so far; build tested only at this point. I think these three patches are ready to queue for 2.6.28, and I don't see any fundamental issue with merging the rest -- I just want to understand things in detail, and that takes time. - R. From rdreier at cisco.com Wed Jul 30 20:24:37 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 20:24:37 -0700 Subject: [ofa-general] [PATCH] IB/uverbs: Support for XRC In-Reply-To: (Roland Dreier's message of "Wed, 30 Jul 2008 20:23:51 -0700") References: Message-ID: From: Jack Morgenstein Add support for core userspace XRC operations (alloc/dealloc XRC domain, create XRC SRQ), including adding an ABI for marshalling requests and responses. Signed-off-by: Jack Morgenstein Signed-off-by: Roland Dreier --- drivers/infiniband/core/uverbs.h | 4 + drivers/infiniband/core/uverbs_cmd.c | 274 ++++++++++++++++++++++++++++++++- drivers/infiniband/core/uverbs_main.c | 35 +++-- include/rdma/ib_user_verbs.h | 35 ++++- 4 files changed, 334 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index b3ea958..f9c051e 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -141,6 +141,7 @@ extern struct idr ib_uverbs_ah_idr; extern struct idr ib_uverbs_cq_idr; extern struct idr ib_uverbs_qp_idr; extern struct idr ib_uverbs_srq_idr; +extern struct idr ib_uverbs_xrcd_idr; void idr_remove_uobj(struct idr *idp, struct ib_uobject *uobj); @@ -194,5 +195,8 @@ IB_UVERBS_DECLARE_CMD(create_srq); IB_UVERBS_DECLARE_CMD(modify_srq); IB_UVERBS_DECLARE_CMD(query_srq); IB_UVERBS_DECLARE_CMD(destroy_srq); +IB_UVERBS_DECLARE_CMD(create_xrc_srq); +IB_UVERBS_DECLARE_CMD(open_xrcd); +IB_UVERBS_DECLARE_CMD(close_xrcd); #endif /* UVERBS_H */ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 56feab6..72840ef 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -46,6 +46,7 @@ static struct lock_class_key cq_lock_key; static struct lock_class_key qp_lock_key; static struct lock_class_key ah_lock_key; static struct lock_class_key srq_lock_key; +static struct lock_class_key xrcd_lock_key; #define INIT_UDATA(udata, ibuf, obuf, ilen, olen) \ do { \ @@ -254,6 +255,17 @@ static void put_srq_read(struct ib_srq *srq) put_uobj_read(srq->uobject); } +static struct ib_xrcd *idr_read_xrcd(int xrcd_handle, + struct ib_ucontext *context) +{ + return idr_read_obj(&ib_uverbs_xrcd_idr, xrcd_handle, context, 0); +} + +static void put_xrcd_read(struct ib_xrcd *xrcd) +{ + put_uobj_read(xrcd->uobject); +} + ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) @@ -297,6 +309,7 @@ ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, INIT_LIST_HEAD(&ucontext->qp_list); INIT_LIST_HEAD(&ucontext->srq_list); INIT_LIST_HEAD(&ucontext->ah_list); + INIT_LIST_HEAD(&ucontext->xrcd_list); ucontext->closing = 0; resp.num_comp_vectors = file->device->num_comp_vectors; @@ -1026,6 +1039,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, struct ib_srq *srq; struct ib_qp *qp; struct ib_qp_init_attr attr; + struct ib_xrcd *xrcd; int ret; if (out_len < sizeof resp) @@ -1045,13 +1059,17 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, init_uobj(&obj->uevent.uobject, cmd.user_handle, file->ucontext, &qp_lock_key); down_write(&obj->uevent.uobject.mutex); - srq = cmd.is_srq ? idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; + srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? + idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; + xrcd = cmd.qp_type == IB_QPT_XRC ? + idr_read_xrcd(cmd.srq_handle, file->ucontext) : NULL; pd = idr_read_pd(cmd.pd_handle, file->ucontext); scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? scq : idr_read_cq(cmd.recv_cq_handle, file->ucontext, 1); - if (!pd || !scq || !rcq || (cmd.is_srq && !srq)) { + if (!pd || !scq || !rcq || (cmd.is_srq && !srq) || + (cmd.qp_type == IB_QPT_XRC && !xrcd)) { ret = -EINVAL; goto err_put; } @@ -1063,6 +1081,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.srq = srq; attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; + attr.xrcd = xrcd; attr.create_flags = 0; attr.cap.max_send_wr = cmd.max_send_wr; @@ -1090,11 +1109,14 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, qp->event_handler = attr.event_handler; qp->qp_context = attr.qp_context; qp->qp_type = attr.qp_type; + qp->xrcd = attr.xrcd; atomic_inc(&pd->usecnt); atomic_inc(&attr.send_cq->usecnt); atomic_inc(&attr.recv_cq->usecnt); if (attr.srq) atomic_inc(&attr.srq->usecnt); + else if (attr.xrcd) + atomic_inc(&attr.xrcd->usecnt); obj->uevent.uobject.object = qp; ret = idr_add_uobj(&ib_uverbs_qp_idr, &obj->uevent.uobject); @@ -1122,6 +1144,8 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, put_cq_read(rcq); if (srq) put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd); mutex_lock(&file->mutex); list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); @@ -1148,6 +1172,8 @@ err_put: put_cq_read(rcq); if (srq) put_srq_read(srq); + if (xrcd) + put_xrcd_read(xrcd); put_uobj_write(&obj->uevent.uobject); return ret; @@ -2000,6 +2026,8 @@ ssize_t ib_uverbs_create_srq(struct ib_uverbs_file *file, srq->uobject = &obj->uobject; srq->event_handler = attr.event_handler; srq->srq_context = attr.srq_context; + srq->xrc_cq = NULL; + srq->xrcd = NULL; atomic_inc(&pd->usecnt); atomic_set(&srq->usecnt, 0); @@ -2045,6 +2073,134 @@ err: return ret; } +ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_create_xrc_srq cmd; + struct ib_uverbs_create_srq_resp resp; + struct ib_udata udata; + struct ib_uevent_object *obj; + struct ib_pd *pd; + struct ib_srq *srq; + struct ib_cq *xrc_cq; + struct ib_xrcd *xrcd; + struct ib_srq_init_attr attr; + int ret; + + if (out_len < sizeof resp) + return -ENOSPC; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + INIT_UDATA(&udata, buf + sizeof cmd, + (unsigned long) cmd.response + sizeof resp, + in_len - sizeof cmd, out_len - sizeof resp); + + obj = kmalloc(sizeof *obj, GFP_KERNEL); + if (!obj) + return -ENOMEM; + + init_uobj(&obj->uobject, cmd.user_handle, file->ucontext, &srq_lock_key); + down_write(&obj->uobject.mutex); + + pd = idr_read_pd(cmd.pd_handle, file->ucontext); + if (!pd) { + ret = -EINVAL; + goto err; + } + + xrc_cq = idr_read_cq(cmd.xrc_cq, file->ucontext, 0); + if (!xrc_cq) { + ret = -EINVAL; + goto err_put_pd; + } + + xrcd = idr_read_xrcd(cmd.xrcd_handle, file->ucontext); + if (!xrcd) { + ret = -EINVAL; + goto err_put_cq; + } + + attr.event_handler = ib_uverbs_srq_event_handler; + attr.srq_context = file; + attr.attr.max_wr = cmd.max_wr; + attr.attr.max_sge = cmd.max_sge; + attr.attr.srq_limit = cmd.srq_limit; + + obj->events_reported = 0; + INIT_LIST_HEAD(&obj->event_list); + + srq = pd->device->create_xrc_srq(pd, xrc_cq, xrcd, &attr, &udata); + if (IS_ERR(srq)) { + ret = PTR_ERR(srq); + goto err_put; + } + + srq->device = pd->device; + srq->pd = pd; + srq->uobject = &obj->uobject; + srq->event_handler = attr.event_handler; + srq->srq_context = attr.srq_context; + srq->xrc_cq = xrc_cq; + srq->xrcd = xrcd; + atomic_inc(&pd->usecnt); + atomic_inc(&xrc_cq->usecnt); + atomic_inc(&xrcd->usecnt); + + atomic_set(&srq->usecnt, 0); + + obj->uobject.object = srq; + ret = idr_add_uobj(&ib_uverbs_srq_idr, &obj->uobject); + if (ret) + goto err_destroy; + + memset(&resp, 0, sizeof resp); + resp.srq_handle = obj->uobject.id; + resp.max_wr = attr.attr.max_wr; + resp.max_sge = attr.attr.max_sge; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + ret = -EFAULT; + goto err_copy; + } + + put_xrcd_read(xrcd); + put_cq_read(xrc_cq); + put_pd_read(pd); + + mutex_lock(&file->mutex); + list_add_tail(&obj->uobject.list, &file->ucontext->srq_list); + mutex_unlock(&file->mutex); + + obj->uobject.live = 1; + + up_write(&obj->uobject.mutex); + + return in_len; + +err_copy: + idr_remove_uobj(&ib_uverbs_srq_idr, &obj->uobject); + +err_destroy: + ib_destroy_srq(srq); + +err_put: + put_xrcd_read(xrcd); + +err_put_cq: + put_cq_read(xrc_cq); + +err_put_pd: + put_pd_read(pd); + +err: + put_uobj_write(&obj->uobject); + return ret; +} + ssize_t ib_uverbs_modify_srq(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) @@ -2163,3 +2319,117 @@ ssize_t ib_uverbs_destroy_srq(struct ib_uverbs_file *file, return ret ? ret : in_len; } + +ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_open_xrcd cmd; + struct ib_uverbs_open_xrcd_resp resp; + struct ib_udata udata; + struct ib_uobject *uobj; + struct ib_xrcd *xrcd; + int ret; + + if (out_len < sizeof resp) + return -ENOSPC; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + /* file descriptors/inodes not yet implemented */ + if (cmd.fd != -1) + return -ENOSYS; + + INIT_UDATA(&udata, buf + sizeof cmd, + (unsigned long) cmd.response + sizeof resp, + in_len - sizeof cmd, out_len - sizeof resp); + + uobj = kmalloc(sizeof *uobj, GFP_KERNEL); + if (!uobj) + return -ENOMEM; + + init_uobj(uobj, 0, file->ucontext, &xrcd_lock_key); + down_write(&uobj->mutex); + + xrcd = file->device->ib_dev->alloc_xrcd(file->device->ib_dev, + file->ucontext, &udata); + if (IS_ERR(xrcd)) { + ret = PTR_ERR(xrcd); + goto err; + } + + xrcd->uobject = uobj; + xrcd->device = file->device->ib_dev; + atomic_set(&xrcd->usecnt, 0); + + uobj->object = xrcd; + ret = idr_add_uobj(&ib_uverbs_xrcd_idr, uobj); + if (ret) + goto err_idr; + + memset(&resp, 0, sizeof resp); + resp.xrcd_handle = uobj->id; + + if (copy_to_user((void __user *) (unsigned long) cmd.response, + &resp, sizeof resp)) { + ret = -EFAULT; + goto err_copy; + } + + mutex_lock(&file->mutex); + list_add_tail(&uobj->list, &file->ucontext->xrcd_list); + mutex_unlock(&file->mutex); + + uobj->live = 1; + + up_write(&uobj->mutex); + + return in_len; + +err_copy: + idr_remove_uobj(&ib_uverbs_pd_idr, uobj); + +err_idr: + ib_dealloc_xrcd(xrcd); + +err: + put_uobj_write(uobj); + return ret; +} + +ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *file, + const char __user *buf, int in_len, + int out_len) +{ + struct ib_uverbs_close_xrcd cmd; + struct ib_uobject *uobj; + int ret; + + if (copy_from_user(&cmd, buf, sizeof cmd)) + return -EFAULT; + + uobj = idr_write_uobj(&ib_uverbs_xrcd_idr, cmd.xrcd_handle, + file->ucontext); + if (!uobj) + return -EINVAL; + + ret = ib_dealloc_xrcd(uobj->object); + if (!ret) + uobj->live = 0; + + put_uobj_write(uobj); + + if (ret) + return ret; + + idr_remove_uobj(&ib_uverbs_xrcd_idr, uobj); + + mutex_lock(&file->mutex); + list_del(&uobj->list); + mutex_unlock(&file->mutex); + + put_uobj(uobj); + + return in_len; +} diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index aeee856..551e508 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -72,6 +72,7 @@ DEFINE_IDR(ib_uverbs_ah_idr); DEFINE_IDR(ib_uverbs_cq_idr); DEFINE_IDR(ib_uverbs_qp_idr); DEFINE_IDR(ib_uverbs_srq_idr); +DEFINE_IDR(ib_uverbs_xrcd_idr); static spinlock_t map_lock; static struct ib_uverbs_device *dev_table[IB_UVERBS_MAX_DEVICES]; @@ -108,6 +109,9 @@ static ssize_t (*uverbs_cmd_table[])(struct ib_uverbs_file *file, [IB_USER_VERBS_CMD_MODIFY_SRQ] = ib_uverbs_modify_srq, [IB_USER_VERBS_CMD_QUERY_SRQ] = ib_uverbs_query_srq, [IB_USER_VERBS_CMD_DESTROY_SRQ] = ib_uverbs_destroy_srq, + [IB_USER_VERBS_CMD_CREATE_XRC_SRQ] = ib_uverbs_create_xrc_srq, + [IB_USER_VERBS_CMD_OPEN_XRCD] = ib_uverbs_open_xrcd, + [IB_USER_VERBS_CMD_CLOSE_XRCD] = ib_uverbs_close_xrcd, }; static struct vfsmount *uverbs_event_mnt; @@ -211,17 +215,6 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uqp); } - list_for_each_entry_safe(uobj, tmp, &context->cq_list, list) { - struct ib_cq *cq = uobj->object; - struct ib_uverbs_event_file *ev_file = cq->cq_context; - struct ib_ucq_object *ucq = - container_of(uobj, struct ib_ucq_object, uobject); - - idr_remove_uobj(&ib_uverbs_cq_idr, uobj); - ib_destroy_cq(cq); - ib_uverbs_release_ucq(file, ev_file, ucq); - kfree(ucq); - } list_for_each_entry_safe(uobj, tmp, &context->srq_list, list) { struct ib_srq *srq = uobj->object; @@ -234,6 +227,18 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uevent); } + list_for_each_entry_safe(uobj, tmp, &context->cq_list, list) { + struct ib_cq *cq = uobj->object; + struct ib_uverbs_event_file *ev_file = cq->cq_context; + struct ib_ucq_object *ucq = + container_of(uobj, struct ib_ucq_object, uobject); + + idr_remove_uobj(&ib_uverbs_cq_idr, uobj); + ib_destroy_cq(cq); + ib_uverbs_release_ucq(file, ev_file, ucq); + kfree(ucq); + } + /* XXX Free MWs */ list_for_each_entry_safe(uobj, tmp, &context->mr_list, list) { @@ -244,6 +249,14 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uobj); } + list_for_each_entry_safe(uobj, tmp, &context->xrcd_list, list) { + struct ib_xrcd *xrcd = uobj->object; + + idr_remove_uobj(&ib_uverbs_xrcd_idr, uobj); + ib_dealloc_xrcd(xrcd); + kfree(uobj); + } + list_for_each_entry_safe(uobj, tmp, &context->pd_list, list) { struct ib_pd *pd = uobj->object; diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h index a17f771..c9e540c 100644 --- a/include/rdma/ib_user_verbs.h +++ b/include/rdma/ib_user_verbs.h @@ -81,7 +81,10 @@ enum { IB_USER_VERBS_CMD_MODIFY_SRQ, IB_USER_VERBS_CMD_QUERY_SRQ, IB_USER_VERBS_CMD_DESTROY_SRQ, - IB_USER_VERBS_CMD_POST_SRQ_RECV + IB_USER_VERBS_CMD_POST_SRQ_RECV, + IB_USER_VERBS_CMD_CREATE_XRC_SRQ, + IB_USER_VERBS_CMD_OPEN_XRCD, + IB_USER_VERBS_CMD_CLOSE_XRCD }; /* @@ -647,6 +650,18 @@ struct ib_uverbs_create_srq { __u64 driver_data[0]; }; +struct ib_uverbs_create_xrc_srq { + __u64 response; + __u64 user_handle; + __u32 pd_handle; + __u32 max_wr; + __u32 max_sge; + __u32 srq_limit; + __u32 xrcd_handle; + __u32 xrc_cq; + __u64 driver_data[0]; +}; + struct ib_uverbs_create_srq_resp { __u32 srq_handle; __u32 max_wr; @@ -686,4 +701,22 @@ struct ib_uverbs_destroy_srq_resp { __u32 events_reported; }; +struct ib_uverbs_open_xrcd { + __u64 response; + __s32 fd; + __u32 oflags; + __u64 driver_data[0]; +}; + +struct ib_uverbs_open_xrcd_resp { + __u32 xrcd_handle; +}; + +struct ib_uverbs_close_xrcd { + __u64 response; + __u32 xrcd_handle; + __u32 reserved; + __u64 driver_data[0]; +}; + #endif /* IB_USER_VERBS_H */ -- 1.5.6.2 From rdreier at cisco.com Wed Jul 30 20:24:51 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 20:24:51 -0700 Subject: [ofa-general] [PATCH] IB/uverbs: Support for associating XRC domains to inodes In-Reply-To: (Roland Dreier's message of "Wed, 30 Jul 2008 20:23:51 -0700") References: Message-ID: From: Jack Morgenstein Add support for associating an XRC domain to an inode so that XRC domains can be shared between processes. We keep a per-device RB tree of XRCDs (indexed by inode) in the uverbs module, and use struct ib_xrcd's usecnt member to reference count XRCDs so that an XRCD is not freed until the last process with a reference is done with it. Signed-off-by: Jack Morgenstein Signed-off-by: Roland Dreier --- drivers/infiniband/core/uverbs.h | 6 +- drivers/infiniband/core/uverbs_cmd.c | 272 ++++++++++++++++++++++++++++----- drivers/infiniband/core/uverbs_main.c | 6 +- drivers/infiniband/core/verbs.c | 2 +- include/rdma/ib_verbs.h | 2 +- 5 files changed, 243 insertions(+), 45 deletions(-) diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index f9c051e..4827fa0 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -70,10 +70,12 @@ struct ib_uverbs_device { struct kref ref; struct completion comp; - int devnum; struct cdev *cdev; struct device *dev; struct ib_device *ib_dev; + struct rb_root xrcd_tree; + struct mutex xrcd_tree_mutex; + int devnum; int num_comp_vectors; }; @@ -161,6 +163,8 @@ void ib_uverbs_qp_event_handler(struct ib_event *event, void *context_ptr); void ib_uverbs_srq_event_handler(struct ib_event *event, void *context_ptr); void ib_uverbs_event_handler(struct ib_event_handler *handler, struct ib_event *event); +void ib_uverbs_dealloc_xrcd(struct ib_uverbs_device *dev, + struct ib_xrcd *xrcd); #define IB_UVERBS_DECLARE_CMD(name) \ ssize_t ib_uverbs_##name(struct ib_uverbs_file *file, \ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 72840ef..d156d8b 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -256,14 +256,11 @@ static void put_srq_read(struct ib_srq *srq) } static struct ib_xrcd *idr_read_xrcd(int xrcd_handle, - struct ib_ucontext *context) + struct ib_ucontext *context, + struct ib_uobject **uobj) { - return idr_read_obj(&ib_uverbs_xrcd_idr, xrcd_handle, context, 0); -} - -static void put_xrcd_read(struct ib_xrcd *xrcd) -{ - put_uobj_read(xrcd->uobject); + *uobj = idr_read_uobj(&ib_uverbs_xrcd_idr, xrcd_handle, context, 0); + return *uobj ? (*uobj)->object : NULL; } ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, @@ -1040,6 +1037,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, struct ib_qp *qp; struct ib_qp_init_attr attr; struct ib_xrcd *xrcd; + struct ib_uobject *xrcd_uobj; int ret; if (out_len < sizeof resp) @@ -1062,7 +1060,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, srq = (cmd.is_srq && cmd.qp_type != IB_QPT_XRC) ? idr_read_srq(cmd.srq_handle, file->ucontext) : NULL; xrcd = cmd.qp_type == IB_QPT_XRC ? - idr_read_xrcd(cmd.srq_handle, file->ucontext) : NULL; + idr_read_xrcd(cmd.srq_handle, file->ucontext, &xrcd_uobj) : NULL; pd = idr_read_pd(cmd.pd_handle, file->ucontext); scq = idr_read_cq(cmd.send_cq_handle, file->ucontext, 0); rcq = cmd.recv_cq_handle == cmd.send_cq_handle ? @@ -1145,7 +1143,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, if (srq) put_srq_read(srq); if (xrcd) - put_xrcd_read(xrcd); + put_uobj_read(xrcd_uobj); mutex_lock(&file->mutex); list_add_tail(&obj->uevent.uobject.list, &file->ucontext->qp_list); @@ -1173,7 +1171,7 @@ err_put: if (srq) put_srq_read(srq); if (xrcd) - put_xrcd_read(xrcd); + put_uobj_read(xrcd_uobj); put_uobj_write(&obj->uevent.uobject); return ret; @@ -2085,6 +2083,7 @@ ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, struct ib_srq *srq; struct ib_cq *xrc_cq; struct ib_xrcd *xrcd; + struct ib_uobject *xrcd_uobj; struct ib_srq_init_attr attr; int ret; @@ -2117,7 +2116,7 @@ ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, goto err_put_pd; } - xrcd = idr_read_xrcd(cmd.xrcd_handle, file->ucontext); + xrcd = idr_read_xrcd(cmd.xrcd_handle, file->ucontext, &xrcd_uobj); if (!xrcd) { ret = -EINVAL; goto err_put_cq; @@ -2167,7 +2166,7 @@ ssize_t ib_uverbs_create_xrc_srq(struct ib_uverbs_file *file, goto err_copy; } - put_xrcd_read(xrcd); + put_uobj_read(xrcd_uobj); put_cq_read(xrc_cq); put_pd_read(pd); @@ -2188,7 +2187,7 @@ err_destroy: ib_destroy_srq(srq); err_put: - put_xrcd_read(xrcd); + put_uobj_read(xrcd_uobj); err_put_cq: put_cq_read(xrc_cq); @@ -2320,6 +2319,93 @@ ssize_t ib_uverbs_destroy_srq(struct ib_uverbs_file *file, return ret ? ret : in_len; } +struct xrcd_table_entry { + struct rb_node node; + struct ib_xrcd *xrcd; + struct inode *inode; +}; + +static int xrcd_table_insert(struct ib_uverbs_device *dev, + struct inode *inode, + struct ib_xrcd *xrcd) +{ + struct xrcd_table_entry *entry, *scan; + struct rb_node **p = &dev->xrcd_tree.rb_node; + struct rb_node *parent = NULL; + + entry = kmalloc(sizeof *entry, GFP_KERNEL); + if (!entry) + return -ENOMEM; + + entry->xrcd = xrcd; + entry->inode = inode; + + while (*p) { + parent = *p; + scan = rb_entry(parent, struct xrcd_table_entry, node); + + if (inode < scan->inode) + p = &(*p)->rb_left; + else if (inode > scan->inode) + p = &(*p)->rb_right; + else { + kfree(entry); + return -EEXIST; + } + } + + rb_link_node(&entry->node, parent, p); + rb_insert_color(&entry->node, &dev->xrcd_tree); + + igrab(inode); + + return 0; +} + +static struct xrcd_table_entry *xrcd_table_search(struct ib_uverbs_device *dev, + struct inode *inode) +{ + struct xrcd_table_entry *entry; + struct rb_node *p = dev->xrcd_tree.rb_node; + + while (p) { + entry = rb_entry(p, struct xrcd_table_entry, node); + + if (inode < entry->inode) + p = p->rb_left; + else if (inode > entry->inode) + p = p->rb_right; + else + return entry; + } + + return NULL; +} + +static struct ib_xrcd *find_xrcd(struct ib_uverbs_device *dev, struct inode *inode) +{ + struct xrcd_table_entry *entry; + + entry = xrcd_table_search(dev, inode); + if (!entry) + return NULL; + + return entry->xrcd; +} + + +static void xrcd_table_delete(struct ib_uverbs_device *dev, + struct inode *inode) +{ + struct xrcd_table_entry *entry = xrcd_table_search(dev, inode); + + if (entry) { + iput(inode); + rb_erase(&entry->node, &dev->xrcd_tree); + kfree(entry); + } +} + ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *file, const char __user *buf, int in_len, int out_len) @@ -2328,8 +2414,11 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *file, struct ib_uverbs_open_xrcd_resp resp; struct ib_udata udata; struct ib_uobject *uobj; - struct ib_xrcd *xrcd; - int ret; + struct ib_xrcd *xrcd = NULL; + struct file *f = NULL; + struct inode *inode = NULL; + int ret = 0; + int new_xrcd = 0; if (out_len < sizeof resp) return -ENOSPC; @@ -2337,31 +2426,61 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *file, if (copy_from_user(&cmd, buf, sizeof cmd)) return -EFAULT; - /* file descriptors/inodes not yet implemented */ - if (cmd.fd != -1) - return -ENOSYS; - INIT_UDATA(&udata, buf + sizeof cmd, (unsigned long) cmd.response + sizeof resp, in_len - sizeof cmd, out_len - sizeof resp); + mutex_lock(&file->device->xrcd_tree_mutex); + + if (cmd.fd != -1) { + /* search for file descriptor */ + f = fget(cmd.fd); + if (!f) { + ret = -EBADF; + goto err_tree_mutex_unlock; + } + + inode = f->f_dentry->d_inode; + if (!inode) { + ret = -EBADF; + goto err_tree_mutex_unlock; + } + + xrcd = find_xrcd(file->device, inode); + if (!xrcd && !(cmd.oflags & O_CREAT)) { + /* no file descriptor. Need CREATE flag */ + ret = -EAGAIN; + goto err_tree_mutex_unlock; + } + + if (xrcd && cmd.oflags & O_EXCL) { + ret = -EINVAL; + goto err_tree_mutex_unlock; + } + } + uobj = kmalloc(sizeof *uobj, GFP_KERNEL); - if (!uobj) - return -ENOMEM; + if (!uobj) { + ret = -ENOMEM; + goto err_tree_mutex_unlock; + } init_uobj(uobj, 0, file->ucontext, &xrcd_lock_key); down_write(&uobj->mutex); - xrcd = file->device->ib_dev->alloc_xrcd(file->device->ib_dev, - file->ucontext, &udata); - if (IS_ERR(xrcd)) { - ret = PTR_ERR(xrcd); - goto err; - } + if (!xrcd) { + xrcd = file->device->ib_dev->alloc_xrcd(file->device->ib_dev, + file->ucontext, &udata); + if (IS_ERR(xrcd)) { + ret = PTR_ERR(xrcd); + goto err; + } - xrcd->uobject = uobj; - xrcd->device = file->device->ib_dev; - atomic_set(&xrcd->usecnt, 0); + xrcd->inode = inode; + xrcd->device = file->device->ib_dev; + atomic_set(&xrcd->usecnt, 0); + new_xrcd = 1; + } uobj->object = xrcd; ret = idr_add_uobj(&ib_uverbs_xrcd_idr, uobj); @@ -2371,12 +2490,25 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *file, memset(&resp, 0, sizeof resp); resp.xrcd_handle = uobj->id; + if (inode) { + if (new_xrcd) { + /* create new inode/xrcd table entry */ + ret = xrcd_table_insert(file->device, inode, xrcd); + if (ret) + goto err_insert_xrcd; + } + atomic_inc(&xrcd->usecnt); + } + if (copy_to_user((void __user *) (unsigned long) cmd.response, &resp, sizeof resp)) { ret = -EFAULT; goto err_copy; } + if (f) + fput(f); + mutex_lock(&file->mutex); list_add_tail(&uobj->list, &file->ucontext->xrcd_list); mutex_unlock(&file->mutex); @@ -2385,43 +2517,77 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *file, up_write(&uobj->mutex); + mutex_unlock(&file->device->xrcd_tree_mutex); return in_len; err_copy: - idr_remove_uobj(&ib_uverbs_pd_idr, uobj); + if (inode) { + if (new_xrcd) + xrcd_table_delete(file->device, inode); + atomic_dec(&xrcd->usecnt); + } + +err_insert_xrcd: + idr_remove_uobj(&ib_uverbs_xrcd_idr, uobj); err_idr: ib_dealloc_xrcd(xrcd); err: put_uobj_write(uobj); + +err_tree_mutex_unlock: + if (f) + fput(f); + + mutex_unlock(&file->device->xrcd_tree_mutex); + return ret; } ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *file, - const char __user *buf, int in_len, - int out_len) + const char __user *buf, int in_len, + int out_len) { struct ib_uverbs_close_xrcd cmd; struct ib_uobject *uobj; - int ret; + struct ib_xrcd *xrcd = NULL; + struct inode *inode = NULL; + int live; + int ret = 0; if (copy_from_user(&cmd, buf, sizeof cmd)) return -EFAULT; + mutex_lock(&file->device->xrcd_tree_mutex); uobj = idr_write_uobj(&ib_uverbs_xrcd_idr, cmd.xrcd_handle, file->ucontext); - if (!uobj) - return -EINVAL; + if (!uobj) { + ret = -EINVAL; + goto out; + } - ret = ib_dealloc_xrcd(uobj->object); - if (!ret) - uobj->live = 0; + xrcd = uobj->object; + inode = xrcd->inode; + + if (!inode || atomic_dec_and_test(&xrcd->usecnt)) { + ret = ib_dealloc_xrcd(uobj->object); + if (!ret) + uobj->live = 0; + } + + live = uobj->live; + + if (inode && ret) + atomic_inc(&xrcd->usecnt); put_uobj_write(uobj); if (ret) - return ret; + goto out; + + if (inode && !live) + xrcd_table_delete(file->device, inode); idr_remove_uobj(&ib_uverbs_xrcd_idr, uobj); @@ -2431,5 +2597,29 @@ ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *file, put_uobj(uobj); - return in_len; + ret = in_len; + +out: + mutex_unlock(&file->device->xrcd_tree_mutex); + return ret; +} + +void ib_uverbs_dealloc_xrcd(struct ib_uverbs_device *dev, + struct ib_xrcd *xrcd) +{ + struct inode *inode = NULL; + int ret = 0; + + inode = xrcd->inode; + if (inode && !atomic_dec_and_test(&xrcd->usecnt)) + return; + + ret = ib_dealloc_xrcd(xrcd); + + if (inode) { + if (!ret) + xrcd_table_delete(dev, inode); + else + atomic_inc(&xrcd->usecnt); + } } diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 551e508..eea10a1 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -249,13 +249,15 @@ static int ib_uverbs_cleanup_ucontext(struct ib_uverbs_file *file, kfree(uobj); } + mutex_lock(&file->device->xrcd_tree_mutex); list_for_each_entry_safe(uobj, tmp, &context->xrcd_list, list) { struct ib_xrcd *xrcd = uobj->object; idr_remove_uobj(&ib_uverbs_xrcd_idr, uobj); - ib_dealloc_xrcd(xrcd); + ib_uverbs_dealloc_xrcd(file->device, xrcd); kfree(uobj); } + mutex_unlock(&file->device->xrcd_tree_mutex); list_for_each_entry_safe(uobj, tmp, &context->pd_list, list) { struct ib_pd *pd = uobj->object; @@ -755,6 +757,8 @@ static void ib_uverbs_add_one(struct ib_device *device) kref_init(&uverbs_dev->ref); init_completion(&uverbs_dev->comp); + uverbs_dev->xrcd_tree = RB_ROOT; + mutex_init(&uverbs_dev->xrcd_tree_mutex); spin_lock(&map_lock); uverbs_dev->devnum = find_first_zero_bit(dev_map, IB_UVERBS_MAX_DEVICES); diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index b75193c..99f76b6 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -1014,7 +1014,7 @@ struct ib_xrcd *ib_alloc_xrcd(struct ib_device *device) xrcd = device->alloc_xrcd(device, NULL, NULL); if (!IS_ERR(xrcd)) { xrcd->device = device; - xrcd->uobject = NULL; + xrcd->inode = NULL; atomic_set(&xrcd->usecnt, 0); } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 8043af7..5170781 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -848,7 +848,7 @@ struct ib_pd { struct ib_xrcd { struct ib_device *device; - struct ib_uobject *uobject; + struct inode *inode; atomic_t usecnt; /* count all resources */ }; -- 1.5.6.2 From rdreier at cisco.com Wed Jul 30 20:25:33 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 30 Jul 2008 20:25:33 -0700 Subject: [ofa-general] [PATCH] IB/core: XRC base implementation In-Reply-To: (Roland Dreier's message of "Wed, 30 Jul 2008 20:24:37 -0700") References: Message-ID: From: Jack Morgenstein Add the core implementation for XRC ("eXtended reliable connected") transport. XRC provides better scalability by allowing senders to specify which shared receive queue (SRQ) should be used to receive a message, which essentially allows one transport context (QP connection) to serve multiple destinations (as long as they shared an adapter, of course). A few new concepts are introduced to support this: - A new device capability flag, IB_DEVICE_XRC, which low-level drivers set to indicate that a device supports XRC. - A new object type: XRC domains (struct ib_xrcd), and new verbs ib_alloc_xrcd()/ib_dealloc_xrcd(). XRCDs are used to limit which XRC SRQs an incoming message can target. - A new QP type, IB_QPT_XRC, which is used to create QPs that use the XRC transport. Creating XRC QPs requires an XRCD to be specified. - A new verb, ib_create_xrc_srq(), which is used to create XRC SRQs. XRC SRQs have an associated SRQ number (SRQN), which is included in incoming messages to target the message to a given SRQ. Signed-off-by: Jack Morgenstein Signed-off-by: Roland Dreier --- drivers/infiniband/core/verbs.c | 138 +++++++++++++++++++++++++++++++++++++-- include/rdma/ib_verbs.h | 60 ++++++++++++++++- 2 files changed, 190 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index a7da9be..b75193c 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -234,6 +234,8 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd, srq->uobject = NULL; srq->event_handler = srq_init_attr->event_handler; srq->srq_context = srq_init_attr->srq_context; + srq->xrc_cq = NULL; + srq->xrcd = NULL; atomic_inc(&pd->usecnt); atomic_set(&srq->usecnt, 0); } @@ -242,6 +244,36 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd, } EXPORT_SYMBOL(ib_create_srq); +struct ib_srq *ib_create_xrc_srq(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *srq_init_attr) +{ + struct ib_srq *srq; + + if (!pd->device->create_xrc_srq) + return ERR_PTR(-ENOSYS); + + srq = pd->device->create_xrc_srq(pd, xrc_cq, xrcd, srq_init_attr, NULL); + + if (!IS_ERR(srq)) { + srq->device = pd->device; + srq->pd = pd; + srq->uobject = NULL; + srq->event_handler = srq_init_attr->event_handler; + srq->srq_context = srq_init_attr->srq_context; + srq->xrc_cq = xrc_cq; + srq->xrcd = xrcd; + atomic_inc(&pd->usecnt); + atomic_inc(&xrcd->usecnt); + atomic_inc(&xrc_cq->usecnt); + atomic_set(&srq->usecnt, 0); + } + + return srq; +} +EXPORT_SYMBOL(ib_create_xrc_srq); + int ib_modify_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr, enum ib_srq_attr_mask srq_attr_mask) @@ -263,16 +295,25 @@ EXPORT_SYMBOL(ib_query_srq); int ib_destroy_srq(struct ib_srq *srq) { struct ib_pd *pd; + struct ib_cq *xrc_cq; + struct ib_xrcd *xrcd; int ret; if (atomic_read(&srq->usecnt)) return -EBUSY; - pd = srq->pd; + pd = srq->pd; + xrc_cq = srq->xrc_cq; + xrcd = srq->xrcd; ret = srq->device->destroy_srq(srq); - if (!ret) + if (!ret) { atomic_dec(&pd->usecnt); + if (xrc_cq) + atomic_dec(&xrc_cq->usecnt); + if (xrcd) + atomic_dec(&xrcd->usecnt); + } return ret; } @@ -297,11 +338,17 @@ struct ib_qp *ib_create_qp(struct ib_pd *pd, qp->event_handler = qp_init_attr->event_handler; qp->qp_context = qp_init_attr->qp_context; qp->qp_type = qp_init_attr->qp_type; + if (qp->qp_type == IB_QPT_XRC) + qp->xrcd = qp_init_attr->xrcd; + else + qp->xrcd = NULL; atomic_inc(&pd->usecnt); atomic_inc(&qp_init_attr->send_cq->usecnt); atomic_inc(&qp_init_attr->recv_cq->usecnt); if (qp_init_attr->srq) atomic_inc(&qp_init_attr->srq->usecnt); + if (qp->xrcd) + atomic_inc(&qp->xrcd->usecnt); } return qp; @@ -327,6 +374,9 @@ static const struct { [IB_QPT_RC] = (IB_QP_PKEY_INDEX | IB_QP_PORT | IB_QP_ACCESS_FLAGS), + [IB_QPT_XRC] = (IB_QP_PKEY_INDEX | + IB_QP_PORT | + IB_QP_ACCESS_FLAGS), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -349,6 +399,9 @@ static const struct { [IB_QPT_RC] = (IB_QP_PKEY_INDEX | IB_QP_PORT | IB_QP_ACCESS_FLAGS), + [IB_QPT_XRC] = (IB_QP_PKEY_INDEX | + IB_QP_PORT | + IB_QP_ACCESS_FLAGS), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -368,6 +421,12 @@ static const struct { IB_QP_RQ_PSN | IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER), + [IB_QPT_XRC] = (IB_QP_AV | + IB_QP_PATH_MTU | + IB_QP_DEST_QPN | + IB_QP_RQ_PSN | + IB_QP_MAX_DEST_RD_ATOMIC | + IB_QP_MIN_RNR_TIMER), }, .opt_param = { [IB_QPT_UD] = (IB_QP_PKEY_INDEX | @@ -378,6 +437,9 @@ static const struct { [IB_QPT_RC] = (IB_QP_ALT_PATH | IB_QP_ACCESS_FLAGS | IB_QP_PKEY_INDEX), + [IB_QPT_XRC] = (IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_PKEY_INDEX), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -398,6 +460,11 @@ static const struct { IB_QP_RNR_RETRY | IB_QP_SQ_PSN | IB_QP_MAX_QP_RD_ATOMIC), + [IB_QPT_XRC] = (IB_QP_TIMEOUT | + IB_QP_RETRY_CNT | + IB_QP_RNR_RETRY | + IB_QP_SQ_PSN | + IB_QP_MAX_QP_RD_ATOMIC), [IB_QPT_SMI] = IB_QP_SQ_PSN, [IB_QPT_GSI] = IB_QP_SQ_PSN, }, @@ -413,6 +480,11 @@ static const struct { IB_QP_ACCESS_FLAGS | IB_QP_MIN_RNR_TIMER | IB_QP_PATH_MIG_STATE), + [IB_QPT_XRC] = (IB_QP_CUR_STATE | + IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_MIN_RNR_TIMER | + IB_QP_PATH_MIG_STATE), [IB_QPT_SMI] = (IB_QP_CUR_STATE | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_CUR_STATE | @@ -437,6 +509,11 @@ static const struct { IB_QP_ALT_PATH | IB_QP_PATH_MIG_STATE | IB_QP_MIN_RNR_TIMER), + [IB_QPT_XRC] = (IB_QP_CUR_STATE | + IB_QP_ACCESS_FLAGS | + IB_QP_ALT_PATH | + IB_QP_PATH_MIG_STATE | + IB_QP_MIN_RNR_TIMER), [IB_QPT_SMI] = (IB_QP_CUR_STATE | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_CUR_STATE | @@ -449,6 +526,7 @@ static const struct { [IB_QPT_UD] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_UC] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_RC] = IB_QP_EN_SQD_ASYNC_NOTIFY, + [IB_QPT_XRC] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_SMI] = IB_QP_EN_SQD_ASYNC_NOTIFY, [IB_QPT_GSI] = IB_QP_EN_SQD_ASYNC_NOTIFY } @@ -471,6 +549,11 @@ static const struct { IB_QP_ACCESS_FLAGS | IB_QP_MIN_RNR_TIMER | IB_QP_PATH_MIG_STATE), + [IB_QPT_XRC] = (IB_QP_CUR_STATE | + IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_MIN_RNR_TIMER | + IB_QP_PATH_MIG_STATE), [IB_QPT_SMI] = (IB_QP_CUR_STATE | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_CUR_STATE | @@ -499,6 +582,18 @@ static const struct { IB_QP_PKEY_INDEX | IB_QP_MIN_RNR_TIMER | IB_QP_PATH_MIG_STATE), + [IB_QPT_XRC] = (IB_QP_PORT | + IB_QP_AV | + IB_QP_TIMEOUT | + IB_QP_RETRY_CNT | + IB_QP_RNR_RETRY | + IB_QP_MAX_QP_RD_ATOMIC | + IB_QP_MAX_DEST_RD_ATOMIC | + IB_QP_ALT_PATH | + IB_QP_ACCESS_FLAGS | + IB_QP_PKEY_INDEX | + IB_QP_MIN_RNR_TIMER | + IB_QP_PATH_MIG_STATE), [IB_QPT_SMI] = (IB_QP_PKEY_INDEX | IB_QP_QKEY), [IB_QPT_GSI] = (IB_QP_PKEY_INDEX | @@ -583,12 +678,14 @@ int ib_destroy_qp(struct ib_qp *qp) struct ib_pd *pd; struct ib_cq *scq, *rcq; struct ib_srq *srq; + struct ib_xrcd *xrcd; int ret; - pd = qp->pd; - scq = qp->send_cq; - rcq = qp->recv_cq; - srq = qp->srq; + pd = qp->pd; + scq = qp->send_cq; + rcq = qp->recv_cq; + srq = qp->srq; + xrcd = qp->xrcd; ret = qp->device->destroy_qp(qp); if (!ret) { @@ -597,6 +694,8 @@ int ib_destroy_qp(struct ib_qp *qp) atomic_dec(&rcq->usecnt); if (srq) atomic_dec(&srq->usecnt); + if (xrcd) + atomic_dec(&xrcd->usecnt); } return ret; @@ -904,3 +1003,30 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid) return qp->device->detach_mcast(qp, gid, lid); } EXPORT_SYMBOL(ib_detach_mcast); + +struct ib_xrcd *ib_alloc_xrcd(struct ib_device *device) +{ + struct ib_xrcd *xrcd; + + if (!device->alloc_xrcd) + return ERR_PTR(-ENOSYS); + + xrcd = device->alloc_xrcd(device, NULL, NULL); + if (!IS_ERR(xrcd)) { + xrcd->device = device; + xrcd->uobject = NULL; + atomic_set(&xrcd->usecnt, 0); + } + + return xrcd; +} +EXPORT_SYMBOL(ib_alloc_xrcd); + +int ib_dealloc_xrcd(struct ib_xrcd *xrcd) +{ + if (atomic_read(&xrcd->usecnt)) + return -EBUSY; + + return xrcd->device->dealloc_xrcd(xrcd); +} +EXPORT_SYMBOL(ib_dealloc_xrcd); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 936e333..8043af7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -103,6 +103,7 @@ enum ib_device_cap_flags { */ IB_DEVICE_UD_IP_CSUM = (1<<18), IB_DEVICE_UD_TSO = (1<<19), + IB_DEVICE_XRC = (1<<20), IB_DEVICE_MEM_MGT_EXTENSIONS = (1<<21), IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22), }; @@ -551,6 +552,7 @@ enum ib_qp_type { IB_QPT_RC, IB_QPT_UC, IB_QPT_UD, + IB_QPT_XRC, IB_QPT_RAW_IPV6, IB_QPT_RAW_ETY }; @@ -566,6 +568,7 @@ struct ib_qp_init_attr { struct ib_cq *send_cq; struct ib_cq *recv_cq; struct ib_srq *srq; + struct ib_xrcd *xrcd; /* XRC QPs only */ struct ib_qp_cap cap; enum ib_sig_type sq_sig_type; enum ib_qp_type qp_type; @@ -753,6 +756,7 @@ struct ib_send_wr { u32 rkey; } fast_reg; } wr; + u32 xrc_remote_srq_num; /* valid for XRC sends only */ }; struct ib_recv_wr { @@ -814,6 +818,7 @@ struct ib_ucontext { struct list_head qp_list; struct list_head srq_list; struct list_head ah_list; + struct list_head xrcd_list; int closing; }; @@ -841,6 +846,12 @@ struct ib_pd { atomic_t usecnt; /* count all resources */ }; +struct ib_xrcd { + struct ib_device *device; + struct ib_uobject *uobject; + atomic_t usecnt; /* count all resources */ +}; + struct ib_ah { struct ib_device *device; struct ib_pd *pd; @@ -862,10 +873,13 @@ struct ib_cq { struct ib_srq { struct ib_device *device; struct ib_pd *pd; + struct ib_cq *xrc_cq; + struct ib_xrcd *xrcd; struct ib_uobject *uobject; void (*event_handler)(struct ib_event *, void *); void *srq_context; atomic_t usecnt; + u32 xrc_srq_num; }; struct ib_qp { @@ -874,6 +888,7 @@ struct ib_qp { struct ib_cq *send_cq; struct ib_cq *recv_cq; struct ib_srq *srq; + struct ib_xrcd *xrcd; /* XRC QPs only */ struct ib_uobject *uobject; void (*event_handler)(struct ib_event *, void *); void *qp_context; @@ -1130,6 +1145,15 @@ struct ib_device { struct ib_grh *in_grh, struct ib_mad *in_mad, struct ib_mad *out_mad); + struct ib_srq * (*create_xrc_srq)(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *srq_init_attr, + struct ib_udata *udata); + struct ib_xrcd * (*alloc_xrcd)(struct ib_device *device, + struct ib_ucontext *context, + struct ib_udata *udata); + int (*dealloc_xrcd)(struct ib_xrcd *xrcd); struct ib_dma_mapping_ops *dma_ops; @@ -1312,8 +1336,28 @@ int ib_query_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr); int ib_destroy_ah(struct ib_ah *ah); /** - * ib_create_srq - Creates a SRQ associated with the specified protection - * domain. + * ib_create_xrc_srq - Creates an XRC SRQ associated with the specified + * protection domain, completion queue, and XRC domain. + * @pd: The protection domain associated with the SRQ. + * @xrc_cq: The CQ to be associated with the XRC SRQ. + * @xrcd: The XRC domain to be associated with the XRC SRQ. + * @srq_init_attr: A list of initial attributes required to create the + * XRC SRQ. If XRC SRQ creation succeeds, then the attributes are + * updated to the actual capabilities of the created XRC SRQ. + * + * srq_attr->max_wr and srq_attr->max_sge are read the determine the + * requested size of the XRC SRQ, and set to the actual values allocated + * on return. If ib_create_xrc_srq() succeeds, then max_wr and max_sge + * will always be at least as large as the requested values. + */ +struct ib_srq *ib_create_xrc_srq(struct ib_pd *pd, + struct ib_cq *xrc_cq, + struct ib_xrcd *xrcd, + struct ib_srq_init_attr *srq_init_attr); + +/** + * ib_create_srq - Creates an SRQ associated with the specified + * protection domain. * @pd: The protection domain associated with the SRQ. * @srq_init_attr: A list of initial attributes required to create the * SRQ. If SRQ creation succeeds, then the attributes are updated to @@ -2031,4 +2075,16 @@ int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid); */ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid); +/** + * ib_alloc_xrcd - Allocates an XRC domain. + * @device: The device on which to allocate the XRC domain. + */ +struct ib_xrcd *ib_alloc_xrcd(struct ib_device *device); + +/** + * ib_dealloc_xrcd - Deallocates an XRC domain. + * @xrcd: The XRC domain to deallocate. + */ +int ib_dealloc_xrcd(struct ib_xrcd *xrcd); + #endif /* IB_VERBS_H */ -- 1.5.6.2 From dwilder at us.ibm.com Wed Jul 30 22:50:38 2008 From: dwilder at us.ibm.com (David J. Wilder) Date: Wed, 30 Jul 2008 22:50:38 -0700 Subject: [ofa-general] [PATCH] Use vmalloc to alloc the rx_ring Message-ID: <1217483438.29436.11.camel@dumpserver> This patch was built on Roland's git. To prevent failures of the rx_ring allocations when using non-srq and large recv_queue_size (1K or larger). vmalloc is used instead of kcalloc. vmalloc uses order 0 allocations. Signed-off-by: David Wilder --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 13 +++++++++---- 1 files changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 0f2d304..0040dbc 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -202,7 +202,7 @@ static void ipoib_cm_free_rx_ring(struct net_device *dev, dev_kfree_skb_any(rx_ring[i].skb); } - kfree(rx_ring); + vfree(rx_ring); } static void ipoib_cm_start_rx_drain(struct ipoib_dev_priv *priv) @@ -352,9 +352,13 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i int ret; int i; - rx->rx_ring = kcalloc(ipoib_recvq_size, sizeof *rx->rx_ring, GFP_KERNEL); - if (!rx->rx_ring) + rx->rx_ring = vmalloc( ipoib_recvq_size * sizeof *rx->rx_ring); + + if (!rx->rx_ring){ + printk(KERN_WARNING "ipoib_cm:Allocation of rx_ring failed, %s", + "try using a lower value of recv_queue_size.\n"); return -ENOMEM; + } t = kmalloc(sizeof *t, GFP_KERNEL); if (!t) { @@ -1494,7 +1498,7 @@ static void ipoib_cm_create_srq(struct net_device *dev, int max_sge) return; } - priv->cm.srq_ring = kzalloc(ipoib_recvq_size * sizeof *priv->cm.srq_ring, + priv->cm.srq_ring = vmalloc(ipoib_recvq_size * sizeof *priv->cm.srq_ring); GFP_KERNEL); if (!priv->cm.srq_ring) { printk(KERN_WARNING "%s: failed to allocate CM SRQ ring (%d entries)\n", @@ -1502,6 +1506,7 @@ static void ipoib_cm_create_srq(struct net_device *dev, int max_sge) ib_destroy_srq(priv->cm.srq); priv->cm.srq = NULL; } + memset(rx_ring, 0, ipoib_recvq_size * sizeof *priv->cm.srq_ring); } int ipoib_cm_dev_init(struct net_device *dev) From ogerlitz at voltaire.com Wed Jul 30 23:38:24 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 31 Jul 2008 09:38:24 +0300 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: References: Message-ID: <48915DE0.3060400@voltaire.com> Hoang-Nam Nguyen wrote: >> If ehca does not generate flush error completions for pending work requests when a QP is transitioned to the error state, then that is a major problem that is going to break cleanup for nearly everything. Generating flush error completions on transition >> to error is a fundamental part of the IB verbs. >> > Right. We're working on a patch for ehca addressing this (flush error cqes) and will post it here for review, probably in 2-3 weeks. So is it a driver, firmware or hardware problem? Or. From Calertleslieleslie at skynet.com.my Thu Jul 31 16:56:41 2008 From: Calertleslieleslie at skynet.com.my (kenneth neville) Date: Thu, 31 Jul 2008 23:56:41 +0000 Subject: [ofa-general] To: general Message-ID: <000701c8f378$0785364d$fcc9dbb8@xxkbqody> Get the great discounts on popular software today ! All sofware is instantly available to download - No Need Wait! ALL OUR SOFTWARES ON ALL EUROPEAN LANGUAGES - USA, English, France, Italy, Spanish, German and more!!! Windows XP Pro With SP2 - $59.95 Adobe Acrobat Pro 8 - $69.95 Office 2003 Pro - $59.95 Adobe Photoshop CS2 - $79.95 AutoCAD 2007 - $149.95 Also we have so much soft for MACINTOSH!!! Microsoft Office 2004 for MAC $79.95 Adobe Acrobat 7 Professional for MAC $59.95 Adobe Creative Suite 2 Premium for MAC $229.95 Macromedia Dreamweaver 8 for MAC $69.95 - Visit our site: www.violerasso[DOT]com (copy this link and replace "[DOT]" to ".") From ogerlitz at voltaire.com Thu Jul 31 00:24:03 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 31 Jul 2008 10:24:03 +0300 Subject: [ofa-general] [PATCH/RFC] RDMA/cma: Remove padding arrays by using struct sockaddr_storage In-Reply-To: References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: <48916893.2050600@voltaire.com> Roland Dreier wrote: > There are a few places that the RDMA CM code handles IPv6 by doing > > struct sockaddr addr; > u8 pad[sizeof(struct sockaddr_in6) - > sizeof(struct sockaddr)]; > > This is fragile and ugly; handle this in a better way with just > > struct sockaddr_storage addr; Looks good to me. There's one more pad usage in ucma.c which I think you want to add into your patch. Also there are few in addr.c which lack the pad but could be handled through Aleksey's patches for IPv6 support. Or From sashak at voltaire.com Thu Jul 31 00:49:15 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 31 Jul 2008 10:49:15 +0300 Subject: [ofa-general] [PATCH] opensm/osm_sminfo_rcv.c: improve locking Message-ID: <20080731074915.GT16811@sashak.voltaire.com> Minor locking improvement. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_sminfo_rcv.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/opensm/opensm/osm_sminfo_rcv.c b/opensm/opensm/osm_sminfo_rcv.c index ec8e4de..47c346d 100644 --- a/opensm/opensm/osm_sminfo_rcv.c +++ b/opensm/opensm/osm_sminfo_rcv.c @@ -179,22 +179,19 @@ __osm_sminfo_rcv_process_set_request(IN osm_sm_t * sm, CL_ASSERT(p_madw); - /* No real need to grab the lock for this function. */ memset(payload, 0, sizeof(payload)); - /* get the lock */ - CL_PLOCK_EXCL_ACQUIRE(sm->p_lock); - p_smp = osm_madw_get_smp_ptr(p_madw); sm_smi = ib_smp_get_payload_ptr(p_smp); if (p_smp->method != IB_MAD_METHOD_SET) { OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 2F03: " "Unsupported method 0x%X\n", p_smp->method); - CL_PLOCK_RELEASE(sm->p_lock); goto Exit; } + CL_PLOCK_EXCL_ACQUIRE(sm->p_lock); + p_smi->guid = sm->p_subn->sm_port_guid; p_smi->act_count = cl_hton32(sm->p_subn->p_osm->stats.qp0_mads_sent); p_smi->pri_state = (uint8_t) (sm->p_subn->sm_state | -- 1.5.5.1.178.g1f811 From ogerlitz at voltaire.com Thu Jul 31 01:07:53 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 31 Jul 2008 11:07:53 +0300 Subject: [ofa-general] [PATCH/RFC] RDMA/cma: Remove padding arrays by using struct sockaddr_storage In-Reply-To: <48916893.2050600@voltaire.com> References: <1217416924.32115.9.camel@linux-zn6t.site> <48916893.2050600@voltaire.com> Message-ID: <489172D9.4090501@voltaire.com> Or Gerlitz wrote: > There's one more pad usage in ucma.c which I think you want to add > into your patch. my mistake, you have addressed that as well. Or. From HNGUYEN at de.ibm.com Thu Jul 31 01:27:59 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 31 Jul 2008 10:27:59 +0200 Subject: [ofa-general] Questions about IPOIB handle last WQE event In-Reply-To: <48915DE0.3060400@voltaire.com> Message-ID: Or Gerlitz wrote on 31.07.2008 08:38:24: > Hoang-Nam Nguyen wrote: > >> If ehca does not generate flush error completions for pending > work requests when a QP is transitioned to the error state, then > that is a major problem that is going to break cleanup for nearly > everything. Generating flush error completions on transition > >> to error is a fundamental part of the IB verbs. > >> > > Right. We're working on a patch for ehca addressing this (flush > error cqes) and will post it here for review, probably in 2-3 weeks. > So is it a driver, firmware or hardware problem? ehca HW claims flush error cqes to be generated by software. Nam From iuzzolin at nmia.com Thu Jul 31 01:55:36 2008 From: iuzzolin at nmia.com (Harold/Carlyn Iuzzolino) Date: Thu, 31 Jul 2008 02:55:36 -0600 Subject: [ofa-general] More comments about the dma-mapping.h, ipath_dma.c problem Message-ID: <200807310855.m6V8ta824129@gandalf.iuzzolino.com> Dear Openfabrics, (in particular Ralph Campbell ) general at lists.openfabrics.org On Mon, 28 Jul 2008, Ralph Campbell wrote: > After looking at this somewhat, I don't see what the bug is. > The various OFED-1.4 files in: > kernel_addons/backport/*/include/linux/dma-mapping.h > either have or don't have #ifndef __x86_64__ > depending on whether valid_dma_direction() is defined in > include/asm-x86_64/dma-mapping.h or not. > For the kernel-devel-2.6.18-1.2798.fc6.x86_64.rpm, > include/asm-x86_64/dma-mapping.h does define valid_dma_direction() > and OFED-1.4 > kernel_addons/backport/backport/2.6.18_FC6/include/linux/dma-mapping.h > does correctly ifndef __x86_64__ valid_dma_direction(). > > I don't have an FC6 system handy so I can't reproduce this. > > Are you sure that you aren't installing on a patched kernel? > You can check by looking at: > cd /lib/modules/`uname -r`/build/include > grep valid_dma_direction asm-x86_64/dma-mapping.h linux/dma-mapping.h I'm using a clean Fedora 6 with NO UPDATES. I had to install the kernel-devel...rpm as well as some other rpms before I ran the OFED install.pl [root at elrond ~]# ls /lib/modules/ 2.6.18-1.2798.fc6xen <--------- Only one directory [root at elrond ~]# uname -a Linux elrond 2.6.18-1.2798.fc6xen #1 SMP Mon Oct 16 14:59:01 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux [root at elrond ~]# uname -r 2.6.18-1.2798.fc6xen [root at elrond include]# cd /lib/modules/`uname -r`/build/include [root at elrond include]# pwd /lib/modules/2.6.18-1.2798.fc6xen/build/include <----------- that's correct [root at elrond include]# ll lrwxrwxrwx 1 root root 10 Jul 2 08:48 asm -> asm-x86_64 drwxr-xr-x 3 root root 4096 Jul 2 08:48 asm-generic drwxr-xr-x 11 root root 4096 Jul 2 08:48 asm-i386 drwxr-xr-x 3 root root 4096 Jul 2 08:48 asm-x86_64 [root at elrond include]# more asm-x86_64/dma-mapping.h You are correct that the file asm-x86_64/dma-mapping.h DOES have a definition of valid_dma_direction(int dma_direction) and it ISN'T ifndef'd out. ------------------------------------- However, here is the compile command from /tmp/OFED.4905.logs/ofa_kernel.rpmbuild.log of the subroutine ipath_dma.c, the one that has the complaint. gcc -Wp,-MD,/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/.ipath_dma.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.1.1/include -Iinclude/asm/mach-xen -D__KERNEL__ \ -include include/linux/autoconf.h \ -include /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/include/linux/autoconf.h \ -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/ \ <--------------------------------- PROBLEM IS HERE \ \ -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/include \ -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/debug \ -I/usr/local/include/scst \ -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/ulp/srpt \ -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/net/cxgb3 \ -Iinclude \ \ -D__XEN_INTERFACE_VERSION__=0x00030203 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit-function-declaration -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_dma)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/.tmp_ipath_dma.o /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c Notice that there is no -I/lib/modules/2.6.18-1.2798.fc6xen/build/include in the above gcc statement. According to what you said, that directory has the correct dma-mapping.h file. The one that's being used, (see below) has the static inline int valid_dma_direction(int dma_direction) code #ifndef __x86_64__ around it. Now, here's the complaint /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c: In function 'ipath_dma_map_single': /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c:58: error: implicit declaration of function 'valid_dma_direction' make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.o] Error 1 I think the problem is in this include directory -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/ \ <--------------------------------- PROBLEM IS HERE [root at elrond include]# ll /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/ total 28 drwxr-xr-x 7 root root 4096 Jul 24 07:10 . drwxr-xr-x 3 root root 4096 Jul 24 07:10 .. drwxr-xr-x 2 root root 4096 Jul 24 07:10 asm drwxr-xr-x 3 root root 4096 Jul 24 07:10 linux drwxr-xr-x 2 root root 4096 Jul 24 07:10 net drwxr-xr-x 2 root root 4096 Jul 24 07:10 scsi drwxr-xr-x 2 root root 4096 Jul 24 07:10 src ----------------- [root at elrond include]# ll /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h -rw-r--r-- 1 root root 1076 Jul 24 07:10 /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h ----------------- [root at elrond include]# more /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h #include_next #ifndef __x86_64__ /* x86_64 was the only architecture that had valid_dma_direction * before 2.6.18 */ static inline int valid_dma_direction(int dma_direction) { return ((dma_direction == DMA_BIDIRECTIONAL) || (dma_direction == DMA_TO_DEVICE) || (dma_direction == DMA_FROM_DEVICE)); } #endif #endif COMMENT: I don't think the directory /lib/modules/2.6.18-1.2798.fc6xen/build/include or the file /lib/modules/2.6.18-1.2798.fc6xen/build/include/asm-x86_64/dma-mapping.h or /lib/modules/2.6.18-1.2798.fc6xen/build/include/linux/dma-mapping.h is being used at all. Instead, the file /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h is being used and it DOES have the definition #ifndef __x86_64__'d out because this IS a 64 bit machine. Now, this is a clean install of Fedora 6 and NOTHING has been updated, on purpose. I used the install.pl script as is. So there's something in that install script that makes it use the wrong directory for the include files. Carlyn <<< Q: How do you make one burn? <<< A: Differentiate a log fire. <<< From vlad at lists.openfabrics.org Thu Jul 31 02:55:09 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Thu, 31 Jul 2008 02:55:09 -0700 (PDT) Subject: [ofa-general] ofa_1_4_kernel 20080731-0200 daily build status Message-ID: <20080731095509.5C5ADE60DCC@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.26 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.24 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Passed on ppc64 with linux-2.6.19 Failed: From alekseys at voltaire.com Thu Jul 31 03:41:59 2008 From: alekseys at voltaire.com (Aleksey Senin) Date: Thu, 31 Jul 2008 13:41:59 +0300 Subject: [ofa-general] Re: [PATCH/RFC] RDMA/cma: Remove padding arrays by using struct sockaddr_storage In-Reply-To: References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: <1217500919.16460.3.camel@linux-zn6t.site> If so, probably it good idea change also functions like rdma_leave_multicast to obtain sockaddr_storage structure, instead of sockaddr and not use casting. Does it make sense? On Wed, 2008-07-30 at 09:57 -0700, Roland Dreier wrote: > There are a few places that the RDMA CM code handles IPv6 by doing > > struct sockaddr addr; > u8 pad[sizeof(struct sockaddr_in6) - > sizeof(struct sockaddr)]; > > This is fragile and ugly; handle this in a better way with just > > struct sockaddr_storage addr; > > Signed-off-by: Roland Dreier > --- > Any objections to merging the cleanup below? > > > drivers/infiniband/core/cma.c | 4 +--- > drivers/infiniband/core/ucma.c | 10 ++++------ > 2 files changed, 5 insertions(+), 9 deletions(-) > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index e980ff3..a16510b 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -155,9 +155,7 @@ struct cma_multicast { > } multicast; > struct list_head list; > void *context; > - struct sockaddr addr; > - u8 pad[sizeof(struct sockaddr_in6) - > - sizeof(struct sockaddr)]; > + struct sockaddr_storage addr; > }; > > struct cma_work { > diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c > index b41dd26..d5a825f 100644 > --- a/drivers/infiniband/core/ucma.c > +++ b/drivers/infiniband/core/ucma.c > @@ -81,9 +81,7 @@ struct ucma_multicast { > > u64 uid; > struct list_head list; > - struct sockaddr addr; > - u8 pad[sizeof(struct sockaddr_in6) - > - sizeof(struct sockaddr)]; > + struct sockaddr_storage addr; > }; > > struct ucma_event { > @@ -913,7 +911,7 @@ static ssize_t ucma_join_multicast(struct ucma_file *file, > > mc->uid = cmd.uid; > memcpy(&mc->addr, &cmd.addr, sizeof cmd.addr); > - ret = rdma_join_multicast(ctx->cm_id, &mc->addr, mc); > + ret = rdma_join_multicast(ctx->cm_id, (struct sockaddr *) &mc->addr, mc); > if (ret) > goto err2; > > @@ -929,7 +927,7 @@ static ssize_t ucma_join_multicast(struct ucma_file *file, > return 0; > > err3: > - rdma_leave_multicast(ctx->cm_id, &mc->addr); > + rdma_leave_multicast(ctx->cm_id, (struct sockaddr *) &mc->addr); > ucma_cleanup_mc_events(mc); > err2: > mutex_lock(&mut); > @@ -975,7 +973,7 @@ static ssize_t ucma_leave_multicast(struct ucma_file *file, > goto out; > } > > - rdma_leave_multicast(mc->ctx->cm_id, &mc->addr); > + rdma_leave_multicast(mc->ctx->cm_id, (struct sockaddr *) &mc->addr); > mutex_lock(&mc->ctx->file->mut); > ucma_cleanup_mc_events(mc); > list_del(&mc->list); From hal.rosenstock at gmail.com Thu Jul 31 04:33:37 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 31 Jul 2008 07:33:37 -0400 Subject: [ofa-general] [PATCH] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. In-Reply-To: <20080730174017.488f81c1.weiny2@llnl.gov> References: <20080730174017.488f81c1.weiny2@llnl.gov> Message-ID: On Wed, Jul 30, 2008 at 8:40 PM, Ira Weiny wrote: > >From d2e52a8b9de02521b01de3414562f45e476cafbf Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 30 Jul 2008 17:28:30 -0700 > Subject: [PATCH] Add a Node Description check on light sweep to ensure that the ND has been > found for each node. This case covers the condition where a ND message is > dropped/lost for some reason and OpenSM is left with a valid configured node > which is not named correctly. A couple of nits below. > This is not the same as a node which has changed it's Node Descriptioin. In > this case the node needs to send a trap. Is that case currently handled ? > Signed-off-by: Ira K. Weiny > --- > opensm/opensm/osm_state_mgr.c | 54 +++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 54 insertions(+), 0 deletions(-) > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > index b599582..15124c9 100644 > --- a/opensm/opensm/osm_state_mgr.c > +++ b/opensm/opensm/osm_state_mgr.c > @@ -506,6 +506,54 @@ Exit: > } > > /********************************************************************** > + During a light sweep check each node to see if the node descriptor is valid > + if not issue a ND query. > +**********************************************************************/ > +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, > + IN void *context) > +{ > + osm_physp_t *p_physp = NULL; > + osm_node_t *const p_node = (osm_node_t *) p_object; > + ib_api_status_t status = IB_SUCCESS; > + osm_madw_context_t mad_context; > + osm_sm_t *sm = (osm_sm_t *)context; > + > + OSM_LOG_ENTER(sm->p_log); > + > + CL_ASSERT(p_node); > + > + if (p_node->print_desc && strcmp(p_node->print_desc, "")) > + /* if ND is valid, do nothing */ > + goto exit; > + > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "__osm_state_mgr_get_node_desc: " > + "Unknown node description \"%s\" for node 0x%016" PRIx64 > + ". Reissuing ND query\n", > + p_node->print_desc ? p_node->print_desc : "", > + cl_ntoh64(osm_node_get_node_guid (p_node))); Needs ERR code. > + > + /* get a physp to request from. */ > + p_physp = osm_node_get_any_physp_ptr(p_node); > + > + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); > + > + status = osm_req_get(sm, > + osm_physp_get_dr_path_ptr(p_physp), > + IB_MAD_ATTR_NODE_DESC, > + 0, CL_DISP_MSGID_NONE, &mad_context); > + if (status != IB_SUCCESS) > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "__osm_ni_rcv_get_node_desc: ERR 0D03: " Shouldn't routine name be __osm_state_mgr_get_node_desc here ? Is ERR code right ? -- Hal > + "Failure initiating NodeDescription request (%s)\n", > + ib_get_err_str(status)); > + > +exit: > + OSM_LOG_EXIT(sm->p_log); > +} > + > + > +/********************************************************************** > Initiates a lightweight sweep of the subnet. > Used during normal sweeps after the subnet is up. > **********************************************************************/ > @@ -514,6 +562,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) > ib_api_status_t status = IB_SUCCESS; > osm_bind_handle_t h_bind; > cl_qmap_t *p_sw_tbl; > + cl_qmap_t *p_node_tbl; > cl_map_item_t *p_next; > osm_node_t *p_node; > osm_physp_t *p_physp; > @@ -522,6 +571,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) > OSM_LOG_ENTER(sm->p_log); > > p_sw_tbl = &sm->p_subn->sw_guid_tbl; > + p_node_tbl = &sm->p_subn->node_guid_tbl; > > /* > * First, get the bind handle. > @@ -540,6 +590,10 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) > cl_qmap_apply_func(p_sw_tbl, __osm_state_mgr_get_sw_info, sm); > CL_PLOCK_RELEASE(sm->p_lock); > > + CL_PLOCK_ACQUIRE(sm->p_lock); > + cl_qmap_apply_func(p_node_tbl, __osm_state_mgr_get_node_desc, sm); > + CL_PLOCK_RELEASE(sm->p_lock); > + > /* now scan the list of physical ports that were not down but have no remote port */ > CL_PLOCK_ACQUIRE(sm->p_lock); > p_next = cl_qmap_head(&sm->p_subn->node_guid_tbl); > -- > 1.5.4.5 > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From rdreier at cisco.com Thu Jul 31 07:27:58 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 31 Jul 2008 07:27:58 -0700 Subject: [ofa-general] Re: [PATCH/RFC] RDMA/cma: Remove padding arrays by using struct sockaddr_storage In-Reply-To: <1217500919.16460.3.camel@linux-zn6t.site> (Aleksey Senin's message of "Thu, 31 Jul 2008 13:41:59 +0300") References: <1217416924.32115.9.camel@linux-zn6t.site> <1217500919.16460.3.camel@linux-zn6t.site> Message-ID: > If so, probably it good idea change also functions like > rdma_leave_multicast to obtain sockaddr_storage structure, instead of > sockaddr and not use casting. Does it make sense? No, I don't think so. API functions use struct sockaddr, struct sockaddr_storage is really just for allocation. - R. From dwilder at us.ibm.com Thu Jul 31 08:43:49 2008 From: dwilder at us.ibm.com (David J. Wilder) Date: Thu, 31 Jul 2008 08:43:49 -0700 Subject: [ofa-general] [PATCH] Use vmalloc to alloc the rx_ring In-Reply-To: <1217483438.29436.11.camel@dumpserver> References: <1217483438.29436.11.camel@dumpserver> Message-ID: <1217519029.9162.3.camel@dumpserver> On Wed, 2008-07-30 at 22:50 -0700, David J. Wilder wrote: > This patch was built on Roland's git. > > To prevent failures of the rx_ring allocations when using non-srq > and large recv_queue_size (1K or larger). > vmalloc is used instead of kcalloc. vmalloc uses order 0 allocations. > > I found a line that should have been removed from the original patch, This one is fixed. To prevent failures of rx_ring when using non-srq and large recv_queue_size (1K or larger) use vmalloc instead of kcalloc when alocation the rx_ring. Signed-off-by: David Wilder --- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 17 ++++++++++++----- 1 files changed, 12 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 0f2d304..2c80f02 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -202,7 +202,7 @@ static void ipoib_cm_free_rx_ring(struct net_device *dev, dev_kfree_skb_any(rx_ring[i].skb); } - kfree(rx_ring); + vfree(rx_ring); } static void ipoib_cm_start_rx_drain(struct ipoib_dev_priv *priv) @@ -352,9 +352,13 @@ static int ipoib_cm_nonsrq_init_rx(struct net_device *dev, struct ib_cm_id *cm_i int ret; int i; - rx->rx_ring = kcalloc(ipoib_recvq_size, sizeof *rx->rx_ring, GFP_KERNEL); - if (!rx->rx_ring) + rx->rx_ring = vmalloc( ipoib_recvq_size * sizeof *rx->rx_ring); + + if (!rx->rx_ring){ + printk(KERN_WARNING "ipoib_cm:Allocation of rx_ring failed, %s", + "try using a lower value of recv_queue_size.\n"); return -ENOMEM; + } t = kmalloc(sizeof *t, GFP_KERNEL); if (!t) { @@ -1494,14 +1498,17 @@ static void ipoib_cm_create_srq(struct net_device *dev, int max_sge) return; } - priv->cm.srq_ring = kzalloc(ipoib_recvq_size * sizeof *priv->cm.srq_ring, - GFP_KERNEL); + priv->cm.srq_ring = vmalloc(ipoib_recvq_size * sizeof *priv->cm.srq_ring); + if (!priv->cm.srq_ring) { printk(KERN_WARNING "%s: failed to allocate CM SRQ ring (%d entries)\n", priv->ca->name, ipoib_recvq_size); ib_destroy_srq(priv->cm.srq); priv->cm.srq = NULL; } + memset(priv->cm.srq_ring, 0, + ipoib_recvq_size * sizeof *priv->cm.srq_ring); + } int ipoib_cm_dev_init(struct net_device *dev) From weiny2 at llnl.gov Thu Jul 31 10:05:23 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 31 Jul 2008 10:05:23 -0700 Subject: [ofa-general] [PATCH V2] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. In-Reply-To: References: <20080730174017.488f81c1.weiny2@llnl.gov> Message-ID: <20080731100523.3f97c6fc.weiny2@llnl.gov> On Thu, 31 Jul 2008 07:33:37 -0400 "Hal Rosenstock" wrote: > On Wed, Jul 30, 2008 at 8:40 PM, Ira Weiny wrote: > > >From d2e52a8b9de02521b01de3414562f45e476cafbf Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Wed, 30 Jul 2008 17:28:30 -0700 > > Subject: [PATCH] Add a Node Description check on light sweep to ensure that the ND has been > > found for each node. This case covers the condition where a ND message is > > dropped/lost for some reason and OpenSM is left with a valid configured node > > which is not named correctly. > > A couple of nits below. > > > This is not the same as a node which has changed it's Node Descriptioin. In > > this case the node needs to send a trap. > > Is that case currently handled ? In OpenSM, yes, Sasha accepted my patches a while ago. I have not gotten around to changing the kernel code to send the trap. However, there is the diag test utility which will send this trap if one wanted to use it. > > > Signed-off-by: Ira K. Weiny > > --- > > opensm/opensm/osm_state_mgr.c | 54 +++++++++++++++++++++++++++++++++++++++++ > > 1 files changed, 54 insertions(+), 0 deletions(-) > > > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > > index b599582..15124c9 100644 > > --- a/opensm/opensm/osm_state_mgr.c > > +++ b/opensm/opensm/osm_state_mgr.c > > @@ -506,6 +506,54 @@ Exit: > > } > > > > /********************************************************************** > > + During a light sweep check each node to see if the node descriptor is valid > > + if not issue a ND query. > > +**********************************************************************/ > > +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, > > + IN void *context) > > +{ > > + osm_physp_t *p_physp = NULL; > > + osm_node_t *const p_node = (osm_node_t *) p_object; > > + ib_api_status_t status = IB_SUCCESS; > > + osm_madw_context_t mad_context; > > + osm_sm_t *sm = (osm_sm_t *)context; > > + > > + OSM_LOG_ENTER(sm->p_log); > > + > > + CL_ASSERT(p_node); > > + > > + if (p_node->print_desc && strcmp(p_node->print_desc, "")) > > + /* if ND is valid, do nothing */ > > + goto exit; > > + > > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > > + "__osm_state_mgr_get_node_desc: " > > + "Unknown node description \"%s\" for node 0x%016" PRIx64 > > + ". Reissuing ND query\n", > > + p_node->print_desc ? p_node->print_desc : "", > > + cl_ntoh64(osm_node_get_node_guid (p_node))); > > Needs ERR code. Ah, yes, I forget. Originally I made this a VERBOSE message but later changed it to ERROR. What method do we use for assigning the error codes? > > > + > > + /* get a physp to request from. */ > > + p_physp = osm_node_get_any_physp_ptr(p_node); > > + > > + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); > > + > > + status = osm_req_get(sm, > > + osm_physp_get_dr_path_ptr(p_physp), > > + IB_MAD_ATTR_NODE_DESC, > > + 0, CL_DISP_MSGID_NONE, &mad_context); > > + if (status != IB_SUCCESS) > > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > > + "__osm_ni_rcv_get_node_desc: ERR 0D03: " > > Shouldn't routine name be __osm_state_mgr_get_node_desc here ? Is ERR > code right ? Sorry, copy and paste error, yes it should be changed. It looks like 3314 and 3315 could be used for the error codes. If these are ok, the new patch is below. If not, let me know and I can change for you. Ira >From 6f7ab6c6ffe64b1da76e5ba68babaaa3356c09fe Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 30 Jul 2008 17:28:30 -0700 Subject: [PATCH] Add a Node Description check on light sweep to ensure that the ND has been found for each node. This case covers the condition where a ND message is dropped/lost for some reason and OpenSM is left with a valid configured node which is not named correctly. This is not the same as a node which has changed it's Node Descriptioin. In this case the node needs to send a trap. Signed-off-by: Ira K. Weiny --- opensm/opensm/osm_state_mgr.c | 54 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 54 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index b599582..a769f52 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -506,6 +506,54 @@ Exit: } /********************************************************************** + During a light sweep check each node to see if the node descriptor is valid + if not issue a ND query. +**********************************************************************/ +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, + IN void *context) +{ + osm_physp_t *p_physp = NULL; + osm_node_t *const p_node = (osm_node_t *) p_object; + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t mad_context; + osm_sm_t *sm = (osm_sm_t *)context; + + OSM_LOG_ENTER(sm->p_log); + + CL_ASSERT(p_node); + + if (p_node->print_desc && strcmp(p_node->print_desc, "")) + /* if ND is valid, do nothing */ + goto exit; + + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "__osm_state_mgr_get_node_desc: ERR 3314: " + "Unknown node description \"%s\" for node 0x%016" PRIx64 + ". Reissuing ND query\n", + p_node->print_desc ? p_node->print_desc : "", + cl_ntoh64(osm_node_get_node_guid (p_node))); + + /* get a physp to request from. */ + p_physp = osm_node_get_any_physp_ptr(p_node); + + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); + + status = osm_req_get(sm, + osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_NODE_DESC, + 0, CL_DISP_MSGID_NONE, &mad_context); + if (status != IB_SUCCESS) + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "__osm_state_mgr_get_node_desc: ERR 3315: " + "Failure initiating NodeDescription request (%s)\n", + ib_get_err_str(status)); + +exit: + OSM_LOG_EXIT(sm->p_log); +} + + +/********************************************************************** Initiates a lightweight sweep of the subnet. Used during normal sweeps after the subnet is up. **********************************************************************/ @@ -514,6 +562,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) ib_api_status_t status = IB_SUCCESS; osm_bind_handle_t h_bind; cl_qmap_t *p_sw_tbl; + cl_qmap_t *p_node_tbl; cl_map_item_t *p_next; osm_node_t *p_node; osm_physp_t *p_physp; @@ -522,6 +571,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) OSM_LOG_ENTER(sm->p_log); p_sw_tbl = &sm->p_subn->sw_guid_tbl; + p_node_tbl = &sm->p_subn->node_guid_tbl; /* * First, get the bind handle. @@ -540,6 +590,10 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) cl_qmap_apply_func(p_sw_tbl, __osm_state_mgr_get_sw_info, sm); CL_PLOCK_RELEASE(sm->p_lock); + CL_PLOCK_ACQUIRE(sm->p_lock); + cl_qmap_apply_func(p_node_tbl, __osm_state_mgr_get_node_desc, sm); + CL_PLOCK_RELEASE(sm->p_lock); + /* now scan the list of physical ports that were not down but have no remote port */ CL_PLOCK_ACQUIRE(sm->p_lock); p_next = cl_qmap_head(&sm->p_subn->node_guid_tbl); -- 1.5.4.5 From sean.hefty at intel.com Thu Jul 31 10:12:52 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 31 Jul 2008 10:12:52 -0700 Subject: [ofa-general] RE: [PATCH 1/1 v2] librdmacm: add support for create qp expanded (with changelog this time) In-Reply-To: References: Message-ID: <000601c8f330$aa81c430$8498070a@amr.corp.intel.com> thanks - I will set this aside until all necessary changes are in libibverbs. >Adds a new function: int rdma_create_qp_expanded >which uses the ibv_create_qp_expanded function in libibverbs and uses >it similarly to ibv_create_qp, with the difference of creation flags. > >Changes in v2: >Added compatibility to old libibverbs. > >Signed-off-by: Ron Livne > >diff --git a/include/rdma/rdma_cma.h b/include/rdma/rdma_cma.h >index a516ab8..34c6b9f 100644 >--- a/include/rdma/rdma_cma.h >+++ b/include/rdma/rdma_cma.h >@@ -296,6 +296,10 @@ int rdma_resolve_route(struct rdma_cm_id *id, int >timeout_ms); > int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, > struct ibv_qp_init_attr *qp_init_attr); > >+int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, >+ struct ibv_qp_init_attr *qp_init_attr, >+ uint32_t create_flags); >+ > /** > * rdma_destroy_qp - Deallocate a QP. > * @id: RDMA identifier. >diff --git a/src/cma.c b/src/cma.c >index ecb41bc..4e2da76 100644 >--- a/src/cma.c >+++ b/src/cma.c >@@ -783,33 +783,49 @@ static int ucma_init_ud_qp(struct cma_id_private >*id_priv, struct ibv_qp *qp) > return ibv_modify_qp(qp, &qp_attr, IBV_QP_STATE | IBV_QP_SQ_PSN); > } > >-int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, >- struct ibv_qp_init_attr *qp_init_attr) >+static int rdma_create_qp_common(struct rdma_cm_id *id, struct ibv_pd *pd, >+ struct ibv_qp_init_attr *qp_init_attr, >+ uint32_t create_flags, int expanded) Can we eliminate expanded and just use 'create_flags'? The patch uses spaces in place of tabs, but I can fix that up. > { >- struct cma_id_private *id_priv; >- struct ibv_qp *qp; >- int ret; >- >- id_priv = container_of(id, struct cma_id_private, id); >- if (id->verbs != pd->context) >- return -EINVAL; >- >- qp = ibv_create_qp(pd, qp_init_attr); >- if (!qp) >- return -ENOMEM; >+ struct cma_id_private *id_priv; >+ struct ibv_qp *qp; >+ int ret; >+ >+ id_priv = container_of(id, struct cma_id_private, id); >+ if (id->verbs != pd->context) >+ return -EINVAL; >+ >+ qp = expanded ? >+ ibv_create_qp_expanded(pd, qp_init_attr, create_flags) : >+ ibv_create_qp(pd, qp_init_attr); >+ if (!qp) >+ return -ENOMEM; >+ >+ if (ucma_is_ud_ps(id->ps)) >+ ret = ucma_init_ud_qp(id_priv, qp); >+ else >+ ret = ucma_init_conn_qp(id_priv, qp); >+ if (ret) >+ goto err; >+ >+ id->qp = qp; >+ return 0; >+err: >+ ibv_destroy_qp(qp); >+ return ret; >+} > >- if (ucma_is_ud_ps(id->ps)) >- ret = ucma_init_ud_qp(id_priv, qp); >- else >- ret = ucma_init_conn_qp(id_priv, qp); >- if (ret) >- goto err; >+int rdma_create_qp_expanded(struct rdma_cm_id *id, struct ibv_pd *pd, >+ struct ibv_qp_init_attr *qp_init_attr, >+ uint32_t create_flags) >+{ >+ return rdma_create_qp_common(id, pd, qp_init_attr, create_flags, 1); >+} > >- id->qp = qp; >- return 0; >-err: >- ibv_destroy_qp(qp); >- return ret; >+int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd, >+ struct ibv_qp_init_attr *qp_init_attr) >+{ >+ return rdma_create_qp_common(id, pd, qp_init_attr, 0, 0); > } > > void rdma_destroy_qp(struct rdma_cm_id *id) >diff --git a/src/librdmacm.map b/src/librdmacm.map >index cb94efe..b237eda 100644 >--- a/src/librdmacm.map >+++ b/src/librdmacm.map >@@ -28,5 +28,6 @@ RDMACM_1.0 { > rdma_get_local_addr; > rdma_get_peer_addr; > rdma_migrate_id; >+ rdma_create_qp_expanded; > local: *; > }; From hal.rosenstock at gmail.com Thu Jul 31 10:18:19 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 31 Jul 2008 13:18:19 -0400 Subject: [ofa-general] [PATCH V2] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. In-Reply-To: <20080731100523.3f97c6fc.weiny2@llnl.gov> References: <20080730174017.488f81c1.weiny2@llnl.gov> <20080731100523.3f97c6fc.weiny2@llnl.gov> Message-ID: On Thu, Jul 31, 2008 at 1:05 PM, Ira Weiny wrote: > On Thu, 31 Jul 2008 07:33:37 -0400 > "Hal Rosenstock" wrote: > >> On Wed, Jul 30, 2008 at 8:40 PM, Ira Weiny wrote: >> > >From d2e52a8b9de02521b01de3414562f45e476cafbf Mon Sep 17 00:00:00 2001 >> > From: Ira K. Weiny >> > Date: Wed, 30 Jul 2008 17:28:30 -0700 >> > Subject: [PATCH] Add a Node Description check on light sweep to ensure that the ND has been >> > found for each node. This case covers the condition where a ND message is >> > dropped/lost for some reason and OpenSM is left with a valid configured node >> > which is not named correctly. >> >> A couple of nits below. >> >> > This is not the same as a node which has changed it's Node Descriptioin. In >> > this case the node needs to send a trap. >> >> Is that case currently handled ? > > In OpenSM, yes, Sasha accepted my patches a while ago. I have not gotten > around to changing the kernel code to send the trap. > However, there is the > diag test utility which will send this trap if one wanted to use it. > >> >> > Signed-off-by: Ira K. Weiny >> > --- >> > opensm/opensm/osm_state_mgr.c | 54 +++++++++++++++++++++++++++++++++++++++++ >> > 1 files changed, 54 insertions(+), 0 deletions(-) >> > >> > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c >> > index b599582..15124c9 100644 >> > --- a/opensm/opensm/osm_state_mgr.c >> > +++ b/opensm/opensm/osm_state_mgr.c >> > @@ -506,6 +506,54 @@ Exit: >> > } >> > >> > /********************************************************************** >> > + During a light sweep check each node to see if the node descriptor is valid >> > + if not issue a ND query. >> > +**********************************************************************/ >> > +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, >> > + IN void *context) >> > +{ >> > + osm_physp_t *p_physp = NULL; >> > + osm_node_t *const p_node = (osm_node_t *) p_object; >> > + ib_api_status_t status = IB_SUCCESS; >> > + osm_madw_context_t mad_context; >> > + osm_sm_t *sm = (osm_sm_t *)context; >> > + >> > + OSM_LOG_ENTER(sm->p_log); >> > + >> > + CL_ASSERT(p_node); >> > + >> > + if (p_node->print_desc && strcmp(p_node->print_desc, "")) >> > + /* if ND is valid, do nothing */ >> > + goto exit; >> > + >> > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, >> > + "__osm_state_mgr_get_node_desc: " >> > + "Unknown node description \"%s\" for node 0x%016" PRIx64 >> > + ". Reissuing ND query\n", >> > + p_node->print_desc ? p_node->print_desc : "", >> > + cl_ntoh64(osm_node_get_node_guid (p_node))); >> >> Needs ERR code. > > Ah, yes, I forget. Originally I made this a VERBOSE message but later changed it > to ERROR. What method do we use for assigning the error codes? Looks like you answered your own question below. >> > + >> > + /* get a physp to request from. */ >> > + p_physp = osm_node_get_any_physp_ptr(p_node); >> > + >> > + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); >> > + >> > + status = osm_req_get(sm, >> > + osm_physp_get_dr_path_ptr(p_physp), >> > + IB_MAD_ATTR_NODE_DESC, >> > + 0, CL_DISP_MSGID_NONE, &mad_context); >> > + if (status != IB_SUCCESS) >> > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, >> > + "__osm_ni_rcv_get_node_desc: ERR 0D03: " >> >> Shouldn't routine name be __osm_state_mgr_get_node_desc here ? Is ERR >> code right ? > > Sorry, copy and paste error, yes it should be changed. > > It looks like 3314 and 3315 could be used for the error codes. If these are > ok, the new patch is below. If not, let me know and I can change for you. 3314 looks available but not 3315. -- Hal > Ira > > > From 6f7ab6c6ffe64b1da76e5ba68babaaa3356c09fe Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 30 Jul 2008 17:28:30 -0700 > Subject: [PATCH] Add a Node Description check on light sweep to ensure that the ND has been > found for each node. This case covers the condition where a ND message is > dropped/lost for some reason and OpenSM is left with a valid configured node > which is not named correctly. > > This is not the same as a node which has changed it's Node Descriptioin. In > this case the node needs to send a trap. > > Signed-off-by: Ira K. Weiny > --- > opensm/opensm/osm_state_mgr.c | 54 +++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 54 insertions(+), 0 deletions(-) > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > index b599582..a769f52 100644 > --- a/opensm/opensm/osm_state_mgr.c > +++ b/opensm/opensm/osm_state_mgr.c > @@ -506,6 +506,54 @@ Exit: > } > > /********************************************************************** > + During a light sweep check each node to see if the node descriptor is valid > + if not issue a ND query. > +**********************************************************************/ > +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, > + IN void *context) > +{ > + osm_physp_t *p_physp = NULL; > + osm_node_t *const p_node = (osm_node_t *) p_object; > + ib_api_status_t status = IB_SUCCESS; > + osm_madw_context_t mad_context; > + osm_sm_t *sm = (osm_sm_t *)context; > + > + OSM_LOG_ENTER(sm->p_log); > + > + CL_ASSERT(p_node); > + > + if (p_node->print_desc && strcmp(p_node->print_desc, "")) > + /* if ND is valid, do nothing */ > + goto exit; > + > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "__osm_state_mgr_get_node_desc: ERR 3314: " > + "Unknown node description \"%s\" for node 0x%016" PRIx64 > + ". Reissuing ND query\n", > + p_node->print_desc ? p_node->print_desc : "", > + cl_ntoh64(osm_node_get_node_guid (p_node))); > + > + /* get a physp to request from. */ > + p_physp = osm_node_get_any_physp_ptr(p_node); > + > + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); > + > + status = osm_req_get(sm, > + osm_physp_get_dr_path_ptr(p_physp), > + IB_MAD_ATTR_NODE_DESC, > + 0, CL_DISP_MSGID_NONE, &mad_context); > + if (status != IB_SUCCESS) > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "__osm_state_mgr_get_node_desc: ERR 3315: " > + "Failure initiating NodeDescription request (%s)\n", > + ib_get_err_str(status)); > + > +exit: > + OSM_LOG_EXIT(sm->p_log); > +} > + > + > +/********************************************************************** > Initiates a lightweight sweep of the subnet. > Used during normal sweeps after the subnet is up. > **********************************************************************/ > @@ -514,6 +562,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) > ib_api_status_t status = IB_SUCCESS; > osm_bind_handle_t h_bind; > cl_qmap_t *p_sw_tbl; > + cl_qmap_t *p_node_tbl; > cl_map_item_t *p_next; > osm_node_t *p_node; > osm_physp_t *p_physp; > @@ -522,6 +571,7 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) > OSM_LOG_ENTER(sm->p_log); > > p_sw_tbl = &sm->p_subn->sw_guid_tbl; > + p_node_tbl = &sm->p_subn->node_guid_tbl; > > /* > * First, get the bind handle. > @@ -540,6 +590,10 @@ static ib_api_status_t __osm_state_mgr_light_sweep_start(IN osm_sm_t * sm) > cl_qmap_apply_func(p_sw_tbl, __osm_state_mgr_get_sw_info, sm); > CL_PLOCK_RELEASE(sm->p_lock); > > + CL_PLOCK_ACQUIRE(sm->p_lock); > + cl_qmap_apply_func(p_node_tbl, __osm_state_mgr_get_node_desc, sm); > + CL_PLOCK_RELEASE(sm->p_lock); > + > /* now scan the list of physical ports that were not down but have no remote port */ > CL_PLOCK_ACQUIRE(sm->p_lock); > p_next = cl_qmap_head(&sm->p_subn->node_guid_tbl); > -- > 1.5.4.5 > > > From sashak at voltaire.com Thu Jul 31 10:37:35 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 31 Jul 2008 20:37:35 +0300 Subject: [ofa-general] [PATCH V2] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. In-Reply-To: <20080731100523.3f97c6fc.weiny2@llnl.gov> References: <20080730174017.488f81c1.weiny2@llnl.gov> <20080731100523.3f97c6fc.weiny2@llnl.gov> Message-ID: <20080731173735.GS14872@sashak.voltaire.com> Hi Ira, On 10:05 Thu 31 Jul , Ira Weiny wrote: > > Ah, yes, I forget. Originally I made this a VERBOSE message but later changed it > to ERROR. What method do we use for assigning the error codes? The method is right two digits are per source file and left two are unique error code inside this file. > +static void __osm_state_mgr_get_node_desc(IN cl_map_item_t * const p_object, > + IN void *context) > +{ > + osm_physp_t *p_physp = NULL; > + osm_node_t *const p_node = (osm_node_t *) p_object; > + ib_api_status_t status = IB_SUCCESS; > + osm_madw_context_t mad_context; > + osm_sm_t *sm = (osm_sm_t *)context; > + > + OSM_LOG_ENTER(sm->p_log); > + > + CL_ASSERT(p_node); > + > + if (p_node->print_desc && strcmp(p_node->print_desc, "")) > + /* if ND is valid, do nothing */ > + goto exit; > + > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "__osm_state_mgr_get_node_desc: ERR 3314: " > + "Unknown node description \"%s\" for node 0x%016" PRIx64 > + ". Reissuing ND query\n", > + p_node->print_desc ? p_node->print_desc : "", > + cl_ntoh64(osm_node_get_node_guid (p_node))); OSM_LOG() macro includes function name, so you don't need to specify this explicitly. > + > + /* get a physp to request from. */ > + p_physp = osm_node_get_any_physp_ptr(p_node); > + > + mad_context.nd_context.node_guid = osm_node_get_node_guid(p_node); > + > + status = osm_req_get(sm, > + osm_physp_get_dr_path_ptr(p_physp), > + IB_MAD_ATTR_NODE_DESC, > + 0, CL_DISP_MSGID_NONE, &mad_context); > + if (status != IB_SUCCESS) > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "__osm_state_mgr_get_node_desc: ERR 3315: " Ditto. Sasha From sashak at voltaire.com Thu Jul 31 10:58:43 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 31 Jul 2008 20:58:43 +0300 Subject: [ofa-general] Re: [PATCH] opensm/osm_ucast_lash.c: A couple more conversions to (unsigned) decimal lid format In-Reply-To: References: Message-ID: <20080731175843.GU14872@sashak.voltaire.com> Applied. Thanks. Sasha From sashak at voltaire.com Thu Jul 31 11:03:23 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 31 Jul 2008 21:03:23 +0300 Subject: [ofa-general] Re: [PATCH] ibsim: Add a Node Description query drop error. In-Reply-To: <20080730174011.5d80e036.weiny2@llnl.gov> References: <20080730174011.5d80e036.weiny2@llnl.gov> Message-ID: <20080731180323.GV14872@sashak.voltaire.com> Hi Ira, On 17:40 Wed 30 Jul , Ira Weiny wrote: > I added this to ibsim to test the patch I am sending next: > > [PATCH] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. > > Thanks, > Ira > > > From ec95833fd08204cb67c9803b488c6ef7e9a30792 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 30 Jul 2008 16:16:11 -0700 > Subject: [PATCH] ibsim: Add a Node Description query drop error. Maybe it could be useful to extend existing 'Error' command by adding attribute there, so user will be able to define drop rate per node, port and/or attribute? Sasha > > > Signed-off-by: Ira K. Weiny > --- > ibsim/sim.h | 1 + > ibsim/sim_cmd.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > ibsim/sim_mad.c | 8 ++++++ > 3 files changed, 83 insertions(+), 0 deletions(-) > > diff --git a/ibsim/sim.h b/ibsim/sim.h > index 936bb85..3cd253f 100644 > --- a/ibsim/sim.h > +++ b/ibsim/sim.h > @@ -207,6 +207,7 @@ struct Port { > Node *remotenode; > int remoteport; > int errrate; > + int nddrop; > Node *node; > Portcounters portcounters; > uint16_t *pkey_tbl; > diff --git a/ibsim/sim_cmd.c b/ibsim/sim_cmd.c > index a6aab9d..fbf2ca3 100644 > --- a/ibsim/sim_cmd.c > +++ b/ibsim/sim_cmd.c > @@ -262,6 +262,74 @@ static int do_seterror(FILE * f, char *line) > return set; > } > > +static int do_setnddrop(FILE * f, char *line) > +{ > + Port *port, *e; > + Node *node; > + char *s = line; > + char *nodeid = 0, name[NAMELEN], *sp, *orig = 0; > + int portnum = -1; // def - all ports > + int numports, set = 0, rate = 0; > + > + if (strsep(&s, "\"")) > + orig = strsep(&s, "\""); > + > + if (!s) { > + fprintf(f, "# unlink: bad parameter in \"%s\"\n", line); > + return -1; > + } > + > + nodeid = expand_name(orig, name, &sp); > + if (!sp && *s == '[') > + sp = s + 1; > + > + if (!(node = find_node(nodeid))) { > + fprintf(f, "# nodeid \"%s\" (%s) not found\n", orig, nodeid); > + return -1; > + } > + > + if (sp) { > + portnum = strtoul(sp, 0, 0); > + if (portnum < 1 || portnum > node->numports) { > + fprintf(f, "# bad port number %d at nodeid \"%s\"\n", > + portnum, nodeid); > + return -1; > + } > + } > + > + numports = node->numports; > + > + if (node->type == SWITCH_NODE) > + numports++; // To make the for-loop below run up to last port > + else > + portnum--; > + > + if (portnum >= 0) { > + port = ports + node->portsbase + portnum; > + if (port->nddrop) > + port->nddrop = 0; > + else > + port->nddrop = 1; > + DEBUG("nddrop is %d", port->nddrop); > + printf("Setting NDDrop %s:%d\n", port->node->nodedesc, port->portnum); > + return 1; > + } > + > + for (port = ports + node->portsbase, e = port + numports; port < e; > + port++) { > + port->errrate = rate; > + if (port->nddrop) > + port->nddrop = 0; > + else > + port->nddrop = 1; > + DEBUG("nddrop is %d", port->nddrop); > + printf("Setting NDDrop %s:%d\n", port->node->nodedesc, port->portnum); > + } > + > + return set; > + > +} > + > static int do_unlink(FILE * f, char *line, int clear) > { > Port *port, *e; > @@ -710,6 +778,8 @@ static int dump_help(FILE * f) > fprintf(f, > "\tError \"nodeid\"[port] : set error rate for port/node\n"); > fprintf(f, > + "\tNDError \"nodeid\"[port]: Toggle the dropping of Node Description packets for the port/node\n"); > + fprintf(f, > "\tBaselid \"nodeid\"[port] [lmc] : change port's lid (lmc)\n"); > fprintf(f, "\tVerbose [newlevel] - show/set simulator verbosity\n"); > fprintf(f, "\t\t\t0 - silent\n"); > @@ -773,6 +843,10 @@ int do_cmd(char *buf, FILE *f) > case 'E': > r = do_seterror(f, line); > break; > + case 'n': > + case 'N': > + r = do_setnddrop(f, line); > + break; > case 'c': > case 'C': > r = do_unlink(f, line, 1); > diff --git a/ibsim/sim_mad.c b/ibsim/sim_mad.c > index b8ce2ab..ac3321e 100644 > --- a/ibsim/sim_mad.c > +++ b/ibsim/sim_mad.c > @@ -1188,6 +1188,14 @@ int process_packet(Client * cl, void *p, int size, Client ** dcl) > return sizeof(*r); // forward only > } > > + /* check for node desc drop. */ > + if ((rpc.mgtclass == IB_SMI_CLASS || rpc.mgtclass == IB_SMI_DIRECT_CLASS) && > + rpc.attr.id == IB_ATTR_NODE_DESC && > + port->nddrop) { > + VERB("drop pkt due to nddrop flag %d", port->nddrop); > + goto _dropped; > + } > + > if (port->errrate && (random() % 100) < port->errrate) { > VERB("drop pkt due error rate %d", port->errrate); > goto _dropped; > -- > 1.5.4.5 > From hal.rosenstock at gmail.com Thu Jul 31 11:04:42 2008 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 31 Jul 2008 14:04:42 -0400 Subject: [ofa-general] ibsim guid initialization Message-ID: Sasha, It appears that ibsim algorithmically determines certain GUIDs (port and system image) rather than getting them from the supplied topology file. Shouldn't it parse the topology file for these ? -- Hal From sashak at voltaire.com Thu Jul 31 11:15:16 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 31 Jul 2008 21:15:16 +0300 Subject: [ofa-general] Re: ibsim guid initialization In-Reply-To: References: Message-ID: <20080731181516.GX14872@sashak.voltaire.com> Hi Hal, On 14:04 Thu 31 Jul , Hal Rosenstock wrote: > > It appears that ibsim algorithmically determines certain GUIDs (port > and system image) rather than getting them from the supplied topology > file. Shouldn't it parse the topology file for these ? Seems good for me. Would you care about patch? The only think to remember there is that this is optional and old ibnetdiscover format doesn't have port guids. Sasha From jon at opengridcomputing.com Thu Jul 31 11:55:06 2008 From: jon at opengridcomputing.com (Jon Mason) Date: Thu, 31 Jul 2008 13:55:06 -0500 Subject: [ofa-general] [PATCH] rds: support for IB_DEVICE_LOCAL_DMA_LKEY Message-ID: <20080731185506.GC23411@opengridcomputing.com> For iWARP, there is a limitation where syncs to remote memory need write permission. By allowing remote write, there is a potential security risk where all memory is available to remote clients. By using the local_dma_lkey, this removes the necessity of remote write permission on local memory regions. The patch below converts the usage of dma_mr's to dma_local_lkey and removes the allocation of dma_mr's (if IB_DEVICE_LOCAL_DMA_LKEY is supported). Also, Chelsio has a limitation of not being able to access DMA MR regions that reside in memory greater that 4GB. So using the patch, rds bcopy will work on systems with greater than 4GB RAM. For IB, using local_dma_lkey removes the need for DMA MR allocations (presuming that the driver supports IB_DEVICE_LOCAL_DMA_LKEY). Signed-Off-By: Jon Mason diff --git a/net/rds/ib.c b/net/rds/ib.c index 4ba4805..8fd8de4 100644 --- a/net/rds/ib.c +++ b/net/rds/ib.c @@ -85,6 +85,7 @@ void rds_ib_add_one(struct ib_device *device) spin_lock_init(&rds_ibdev->spinlock); + rds_ibdev->dma_local_lkey = !!(dev_attr->device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY); rds_ibdev->max_wrs = dev_attr->max_qp_wr; rds_ibdev->max_sge = min(dev_attr->max_sge, RDS_IB_MAX_SGE); @@ -103,18 +104,21 @@ void rds_ib_add_one(struct ib_device *device) if (IS_ERR(rds_ibdev->pd)) goto free_dev; - if (device->node_type != RDMA_NODE_RNIC) { - rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, - IB_ACCESS_LOCAL_WRITE); - } else { - /* Why does it have to have these permissions? */ - rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, - IB_ACCESS_REMOTE_READ | - IB_ACCESS_REMOTE_WRITE | - IB_ACCESS_LOCAL_WRITE); - } - if (IS_ERR(rds_ibdev->mr)) - goto err_pd; + if (!rds_ibdev->dma_local_lkey) { + if (device->node_type != RDMA_NODE_RNIC) { + rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, + IB_ACCESS_LOCAL_WRITE); + } else { + /* Why does it have to have these permissions? */ + rds_ibdev->mr = ib_get_dma_mr(rds_ibdev->pd, + IB_ACCESS_REMOTE_READ | + IB_ACCESS_REMOTE_WRITE | + IB_ACCESS_LOCAL_WRITE); + } + if (IS_ERR(rds_ibdev->mr)) + goto err_pd; + } else + rds_ibdev->mr = NULL; /* Tell the RDMA code to use the fastreg API */ if (dev_attr->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS) @@ -134,7 +138,8 @@ void rds_ib_add_one(struct ib_device *device) goto free_attr; err_mr: - ib_dereg_mr(rds_ibdev->mr); + if (!rds_ibdev->dma_local_lkey) + ib_dereg_mr(rds_ibdev->mr); err_pd: ib_dealloc_pd(rds_ibdev->pd); free_dev: diff --git a/net/rds/ib.h b/net/rds/ib.h index 9e17075..13eb1f2 100644 --- a/net/rds/ib.h +++ b/net/rds/ib.h @@ -157,7 +157,8 @@ struct rds_ib_connection { /* Protocol version specific information */ unsigned int i_flowctl : 1, /* enable/disable flow ctl */ i_iwarp : 1, /* this is actually iWARP not IB */ - i_fastreg : 1; /* device supports fastreg */ + i_fastreg : 1, /* device supports fastreg */ + i_dma_local_lkey : 1; /* Batched completions */ unsigned int i_unsignaled_wrs; @@ -189,7 +190,8 @@ struct rds_ib_device { unsigned int max_fmrs; int max_sge; unsigned int max_wrs; - unsigned int use_fastreg : 1; + unsigned int use_fastreg : 1, + dma_local_lkey : 1; spinlock_t spinlock; }; @@ -264,6 +266,10 @@ static void inline rds_ib_dma_sync_sg_for_device(struct ib_device *dev, } #define ib_dma_sync_sg_for_device rds_ib_dma_sync_sg_for_device +static inline u32 rds_ib_local_dma_lkey(struct rds_ib_connection *ic) +{ + return (ic->i_dma_local_lkey ? ic->i_cm_id->device->local_dma_lkey : ic->i_mr->lkey); +} /* ib.c */ extern struct rds_transport rds_ib_transport; diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index 9969504..89be6ba 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -516,6 +516,7 @@ static int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, /* Remember whether this is IB or iWARP */ ic->i_iwarp = (cm_id->device->node_type == RDMA_NODE_RNIC); ic->i_fastreg = rds_ibdev->use_fastreg; + ic->i_dma_local_lkey = rds_ibdev->dma_local_lkey; /* We got halfway through setting up the ib_connection, if we * fail now, we have to take the long route out of this mess. */ @@ -671,6 +672,7 @@ out: int rds_ib_conn_connect(struct rds_connection *conn) { struct rds_ib_connection *ic = conn->c_transport_data; + struct rds_ib_device *rds_ibdev; struct sockaddr_in src, dest; int ret; @@ -698,8 +700,12 @@ int rds_ib_conn_connect(struct rds_connection *conn) goto out; } + rds_ibdev = ib_get_client_data(ic->i_cm_id->device, &rds_ib_client); + /* Now check the device type and set i_iwarp */ ic->i_iwarp = (ic->i_cm_id->device->node_type == RDMA_NODE_RNIC); + ic->i_fastreg = rds_ibdev->use_fastreg; + ic->i_dma_local_lkey = rds_ibdev->dma_local_lkey; dest.sin_family = AF_INET; dest.sin_addr.s_addr = (__force u32)conn->c_faddr; diff --git a/net/rds/ib_recv.c b/net/rds/ib_recv.c index 9f72556..6738758 100644 --- a/net/rds/ib_recv.c +++ b/net/rds/ib_recv.c @@ -97,12 +97,12 @@ void rds_ib_recv_init_ring(struct rds_ib_connection *ic) sge = rds_ib_data_sge(ic, recv->r_sge); sge->addr = 0; sge->length = RDS_FRAG_SIZE; - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); sge = rds_ib_header_sge(ic, recv->r_sge); sge->addr = ic->i_recv_hdrs_dma + (i * sizeof(struct rds_header)); sge->length = sizeof(struct rds_header); - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); } } @@ -364,7 +364,7 @@ void rds_ib_recv_init_ack(struct rds_ib_connection *ic) sge->addr = ic->i_ack_dma; sge->length = sizeof(struct rds_header); - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); wr->sg_list = sge; wr->num_sge = 1; diff --git a/net/rds/ib_send.c b/net/rds/ib_send.c index 4878d3b..6d4e99d 100644 --- a/net/rds/ib_send.c +++ b/net/rds/ib_send.c @@ -145,12 +145,12 @@ void rds_ib_send_init_ring(struct rds_ib_connection *ic) send->s_wr.ex.imm_data = 0; sge = rds_ib_data_sge(ic, send->s_sge); - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); sge = rds_ib_header_sge(ic, send->s_sge); sge->addr = ic->i_send_hdrs_dma + (i * sizeof(struct rds_header)); sge->length = sizeof(struct rds_header); - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); } } @@ -432,7 +432,7 @@ rds_ib_xmit_populate_wr(struct rds_ib_connection *ic, sge = rds_ib_data_sge(ic, send->s_sge); sge->addr = buffer; sge->length = length; - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); sge = rds_ib_header_sge(ic, send->s_sge); } else { @@ -444,7 +444,7 @@ rds_ib_xmit_populate_wr(struct rds_ib_connection *ic, sge->addr = ic->i_send_hdrs_dma + (pos * sizeof(struct rds_header)); sge->length = sizeof(struct rds_header); - sge->lkey = ic->i_mr->lkey; + sge->lkey = rds_ib_local_dma_lkey(ic); } /* @@ -813,7 +813,7 @@ int rds_ib_xmit_rdma(struct rds_connection *conn, struct rds_rdma_op *op) len = sg_dma_len(scat); send->s_sge[j].addr = sg_dma_address(scat); send->s_sge[j].length = len; - send->s_sge[j].lkey = ic->i_mr->lkey; + send->s_sge[j].lkey = rds_ib_local_dma_lkey(ic); sent += len; rdsdebug("ic %p sent %d remote_addr %llu\n", ic, sent, remote_addr); From sean.hefty at intel.com Thu Jul 31 12:06:18 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 31 Jul 2008 12:06:18 -0700 Subject: [ofa-general] RE: [PATCH/RFC] RDMA/cma: Remove padding arrays by using struct sockaddr_storage In-Reply-To: References: <1217416924.32115.9.camel@linux-zn6t.site> Message-ID: <001901c8f340$833452c0$8498070a@amr.corp.intel.com> >There are a few places that the RDMA CM code handles IPv6 by doing > > struct sockaddr addr; > u8 pad[sizeof(struct sockaddr_in6) - > sizeof(struct sockaddr)]; > >This is fragile and ugly; handle this in a better way with just > > struct sockaddr_storage addr; > >Signed-off-by: Roland Dreier >--- >Any objections to merging the cleanup below? nope - thanks From weiny2 at llnl.gov Thu Jul 31 13:20:53 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 31 Jul 2008 13:20:53 -0700 Subject: [ofa-general] Re: [PATCH V2] ibsim: Add a Node Description query drop error. In-Reply-To: <20080731180323.GV14872@sashak.voltaire.com> References: <20080730174011.5d80e036.weiny2@llnl.gov> <20080731180323.GV14872@sashak.voltaire.com> Message-ID: <20080731132053.17cd8915.weiny2@llnl.gov> On Thu, 31 Jul 2008 21:03:23 +0300 Sasha Khapyorsky wrote: > Hi Ira, > > On 17:40 Wed 30 Jul , Ira Weiny wrote: > > I added this to ibsim to test the patch I am sending next: > > > > [PATCH] OpenSM: Add a Node Description check on light sweep to ensure that the ND has been found for each node. > > > > Thanks, > > Ira > > > > > > From ec95833fd08204cb67c9803b488c6ef7e9a30792 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Wed, 30 Jul 2008 16:16:11 -0700 > > Subject: [PATCH] ibsim: Add a Node Description query drop error. > > Maybe it could be useful to extend existing 'Error' command by adding > attribute there, so user will be able to define drop rate per node, port > and/or attribute? > Like this? Ira >From 0fc07721a55a82feaa304ebe15e9a236d4954f02 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 30 Jul 2008 16:16:11 -0700 Subject: [PATCH] Add a Node Description query drop error. Signed-off-by: Ira K. Weiny --- ibsim/sim.h | 1 + ibsim/sim_cmd.c | 46 ++++++++++++++++++++++++++++++++++++---------- ibsim/sim_mad.c | 8 ++++++++ 3 files changed, 45 insertions(+), 10 deletions(-) diff --git a/ibsim/sim.h b/ibsim/sim.h index 936bb85..3cd253f 100644 --- a/ibsim/sim.h +++ b/ibsim/sim.h @@ -207,6 +207,7 @@ struct Port { Node *remotenode; int remoteport; int errrate; + int nddrop; Node *node; Portcounters portcounters; uint16_t *pkey_tbl; diff --git a/ibsim/sim_cmd.c b/ibsim/sim_cmd.c index a6aab9d..0df87bd 100644 --- a/ibsim/sim_cmd.c +++ b/ibsim/sim_cmd.c @@ -191,6 +191,15 @@ static void port_change_lid(Port * port, int lid, int lmc) port->remotenode->sw->portchange = 1; } +#define TOGGLE_NDDROP(portptr) \ + if (portptr->nddrop) \ + portptr->nddrop = 0; \ + else \ + portptr->nddrop = 1; \ + printf("Node Description Drop: %s port %d == %d\n", \ + port->node->nodedesc, port->portnum, \ + port->nddrop) + static int do_seterror(FILE * f, char *line) { Port *port, *e; @@ -198,7 +207,7 @@ static int do_seterror(FILE * f, char *line) char *s = line; char *nodeid = 0, name[NAMELEN], *sp, *orig = 0; int portnum = -1; // def - all ports - int numports, set = 0, rate = 0; + int numports, set = 0, rate = 0, nddrop = -1; if (strsep(&s, "\"")) orig = strsep(&s, "\""); @@ -231,12 +240,18 @@ static int do_seterror(FILE * f, char *line) return -1; } - rate = strtoul(s, 0, 0); - - if (rate > 100) { - fprintf(f, "# error rate must be in [0..100] range (%d)\n", - rate); - return -1; + /* Check for specific error requests */ + if (strcmp(s, "nddrop")) { + nddrop = 1; + rate = -1; + } else { + rate = strtoul(s, 0, 0); + if (rate > 100) { + fprintf(f, "# error rate must be in [0..100] range (%d)\n", + rate); + return -1; + } + nddrop = -1; } DEBUG("error rate is %d", rate); @@ -249,13 +264,21 @@ static int do_seterror(FILE * f, char *line) if (portnum >= 0) { port = ports + node->portsbase + portnum; - port->errrate = rate; + if (nddrop) { + TOGGLE_NDDROP(port); + } else { + port->errrate = rate; + } return 1; } for (port = ports + node->portsbase, e = port + numports; port < e; port++) { - port->errrate = rate; + if (nddrop) { + TOGGLE_NDDROP(port); + } else { + port->errrate = rate; + } set++; } @@ -708,7 +731,10 @@ static int dump_help(FILE * f) fprintf(f, "\tGuid \"nodeid\" : set GUID value for this node\n"); fprintf(f, "\tGuid \"nodeid\"[port] : set GUID value for this port\n"); fprintf(f, - "\tError \"nodeid\"[port] : set error rate for port/node\n"); + "\tError \"nodeid\"[port] [|nddrop]: set errors for port/node\n"); + fprintf(f, "\t\t\t sets a random rate for all packets\n"); + fprintf(f, + "\t\t\t\"nddrop\" Toggles the dropping of Node Description packets only\n"); fprintf(f, "\tBaselid \"nodeid\"[port] [lmc] : change port's lid (lmc)\n"); fprintf(f, "\tVerbose [newlevel] - show/set simulator verbosity\n"); diff --git a/ibsim/sim_mad.c b/ibsim/sim_mad.c index b8ce2ab..ac3321e 100644 --- a/ibsim/sim_mad.c +++ b/ibsim/sim_mad.c @@ -1188,6 +1188,14 @@ int process_packet(Client * cl, void *p, int size, Client ** dcl) return sizeof(*r); // forward only } + /* check for node desc drop. */ + if ((rpc.mgtclass == IB_SMI_CLASS || rpc.mgtclass == IB_SMI_DIRECT_CLASS) && + rpc.attr.id == IB_ATTR_NODE_DESC && + port->nddrop) { + VERB("drop pkt due to nddrop flag %d", port->nddrop); + goto _dropped; + } + if (port->errrate && (random() % 100) < port->errrate) { VERB("drop pkt due error rate %d", port->errrate); goto _dropped; -- 1.5.4.5 From swise at opengridcomputing.com Thu Jul 31 13:21:35 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 31 Jul 2008 15:21:35 -0500 Subject: [ofa-general] [PATCH 2.6.27] RDMA/cxgb3: Fix QP capabilities. Message-ID: <20080731202135.18293.57833.stgit@dell3.ogc.int> From: Steve Wise - Set the stag0 and fastreg capability bits only for kernel qps. - QP_PRIV flag is no longer used. Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/iwch_qp.c | 23 +++++++---------------- 1 files changed, 7 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c index 9a3be3a..8939716 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_qp.c +++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c @@ -879,20 +879,13 @@ static int rdma_init(struct iwch_dev *rhp, struct iwch_qp *qhp, (qhp->attr.mpa_attr.xmit_marker_enabled << 1) | (qhp->attr.mpa_attr.crc_enabled << 2); - /* - * XXX - The IWCM doesn't quite handle getting these - * attrs set before going into RTS. For now, just turn - * them on always... - */ -#if 0 - init_attr.qpcaps = qhp->attr.enableRdmaRead | - (qhp->attr.enableRdmaWrite << 1) | - (qhp->attr.enableBind << 2) | - (qhp->attr.enable_stag0_fastreg << 3) | - (qhp->attr.enable_stag0_fastreg << 4); -#else - init_attr.qpcaps = 0x1f; -#endif + init_attr.qpcaps = uP_RI_QP_RDMA_READ_ENABLE | + uP_RI_QP_RDMA_WRITE_ENABLE | + uP_RI_QP_BIND_ENABLE; + if (!qhp->ibqp.uobject) + init_attr.qpcaps |= uP_RI_QP_STAG0_ENABLE | + uP_RI_QP_FAST_REGISTER_ENABLE; + init_attr.tcp_emss = qhp->ep->emss; init_attr.ord = qhp->attr.max_ord; init_attr.ird = qhp->attr.max_ird; @@ -900,8 +893,6 @@ static int rdma_init(struct iwch_dev *rhp, struct iwch_qp *qhp, init_attr.qp_dma_size = (1UL << qhp->wq.size_log2); init_attr.rqe_count = iwch_rqes_posted(qhp); init_attr.flags = qhp->attr.mpa_attr.initiator ? MPA_INITIATOR : 0; - if (!qhp->ibqp.uobject) - init_attr.flags |= PRIV_QP; if (peer2peer) { init_attr.rtr_type = RTR_READ; if (init_attr.ord == 0 && qhp->attr.mpa_attr.initiator) From cameron at harr.org Thu Jul 31 14:36:42 2008 From: cameron at harr.org (Cameron Harr) Date: Thu, 31 Jul 2008 15:36:42 -0600 Subject: [ofa-general] OFED build problems on Centos 5.2 Message-ID: <4892306A.2000307@harr.org> Vladimir, I too am seeing the same problem when I try to build all packages or something with iSer.: RPM build errors: user vlad does not exist - using root group vlad does not exist - using root user vlad does not exist - using root group vlad does not exist - using root File not found: /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/drivers/scsi/iscsi_tcp.ko File not found: /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/drivers/scsi/libiscsi.ko File not found: /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/drivers/scsi/scsi_transport_iscsi.ko File not found: /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/net/rds File not found: /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/drivers/net/cxgb3 File not found: /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/drivers/net/mlx4 Do you have any update on it, or would you like me to send you my build log? Also, in playing around, I've found that I can compile my general Ofed configuration of mthca drivers and SRPT without errors, but the ib_srpt module comes up missing in updates/kernel/drivers/infiniband/ulp/srpt directory. I've scanned all the built rpms, and it's not there. Oddly enough, when the rpm is being built, I can see it included: + test -e /var/tmp/OFED/lib/modules/2.6.18-92.1.6.el5/updates/kernel/drivers/infiniband/ib_srpt.ko It just seem like it and some of the other modules are never actually packaged. Thanks, Cameron From sean.hefty at intel.com Thu Jul 31 16:16:21 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 31 Jul 2008 16:16:21 -0700 Subject: [ofa-general] [ANNOUNCE] new releases of IB and RDMA CM libraries Message-ID: <002601c8f363$71e9d1c0$8498070a@amr.corp.intel.com> libibcm 1.0.3 and librdmacm 1.0.8 are now available from: http://www.openfabrics.org/downloads/rdmacm/ Most relevant changes: libibcm - Add definitions for obtaining CM assigned service IDs - Fix IB CM access when iWarp cards are present in the system librdmacm - Report address change event (new in kernel 2.6.27) - Report timewait exit event (new in kernel 2.6.27) - Minor enhancements to example programs OFED 1.4: Please use these versions in the newest OFED releases Thanks, - Sean From mack_nick44 at yahoo.com Thu Jul 31 17:13:15 2008 From: mack_nick44 at yahoo.com (From Window Live Awards Centre) Date: Fri, 1 Aug 2008 06:13:15 +0600 Subject: [ofa-general] lucky winner Message-ID: <200808010013.m710DFbu018716@chat.intersat.ru> Yahoo & Window Live Awards Centre 124 Stockport Road, Longsight, Manchester M60 2DB - United Kingdom. This is to inform you that you have won a prize money of one million Great Britain Pounds (GBP 1000,000.00) for the month of June, 2008 Prize promotion which is organized by YAHOO A WARDS & WINDOWS LIVE. YAHOO collects all the email addresses of the people that are active online, among the millions that subscribed to Yahoo and Hotmail and few from other e-mail providers. Six people are selected monthly to benefit from this promotion and you are one of the Selected Winners. PAYMENT OF PRIZE / CLAIMS Winners shall be paid in accordance with their Settlement Centers. Yahoo Prize Award must be claimed not later than 30 days from date of Draw Notification. Any prize not claimed within this period will be forfeited and returned to its source as unclaimed. Stated below are your identification numbers: BATCH NUMBER: MFI/06/APA-43658 REFERENCE NUMBER: 2007234522 PIN: 1207 You are requested to contact our co-ordinator in Manchester and send your winning identification numbers to him: Co-ordinator's Name: Mr. Mack Nick E-Mail:cordinatormack1 at yahoo.fr Tel:(44) 704 577 6061 You are advised to send the following information to our co-ordinator to facilitate the release of of your fund to you. 1. Full name.............................................. 2. Country................................................. 3. Contact Address................................ 4. Telephone Number............................ 5. fax Number.......................................... 5. Marital Status..................................... 6. Occupation.......................................... 7. sex......................................................... 8. Date of Birth/Age ............................... 9. Identity card(carte identite)............. Congratulations!! once again. Yours in service, Mrs. Grace Sanders(Awards coordinator) ----------------------------------------------------------------------- ----------------------------------------------------------------------- ---WARNING! You must keep strict confidentiality of your Prize Award claims to yourself until your money is successfully handed over to you to avoid disqualification that may arise from double claims. Yahoo Awards Team shall not be held liable for any loss of funds arising from the above mentioned. From ralph.campbell at qlogic.com Thu Jul 31 17:25:11 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 31 Jul 2008 17:25:11 -0700 Subject: [ofa-general] More comments about the dma-mapping.h, ipath_dma.c problem In-Reply-To: <200807310855.m6V8ta824129@gandalf.iuzzolino.com> References: <200807310855.m6V8ta824129@gandalf.iuzzolino.com> Message-ID: <1217550311.620.115.camel@chromite.mv.qlogic.com> Thanks for the additional information. I am able to reproduce your compile problem. The problem is that XEN has its own dma-mapping.h file which #if 0 around all the dma_* functions: /usr/src/kernels/2.6.18-1.2798.fc6-xen-x86_64/include/asm-x86_64/mach-xen/asm/dma-mapping.h XEN is not a supported kernel so this has not been tested, but if you want to try it, I did the following: # git clone git://git.openfabrics.org/ofed_1_3/linux-2.6.git # cd linux-2.6.git # ./ofed_scripts/ofed_makedist.sh # tar zxf ofed_kernel-2.6.18_FC6.tgz # cd ofed_kernel-2.6.18_FC6 # vi kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h # ./configure \ --with-core-mod \ --with-ipoib-mod \ --with-ipoib-cm \ --without-ipoib_debug-mod \ --without-ipoib_debug_data-mod \ --with-sdp-mod \ --without-sdp_debug-mod \ --without-sdp_debug_data-mod \ --with-srp-mod \ --with-srp-target-mod \ --with-user_mad-mod \ --with-user_access-mod \ --with-addr_trans-mod \ --with-mthca-mod \ --without-mthca_debug-mod \ --with-mlx4-mod \ --without-mlx4_debug-mod \ --with-ipath_inf-mod \ --without-iser-mod \ --without-ehca-mod \ --with-rds-mod \ --without-rds_debug-mod \ --with-madeye-mod \ --with-qlgc_vnic-mod \ --without-qlgc_vnic_debug-mod \ --without-qlgc_vnic_stats-mod \ --without-cxgb3-mod \ --without-cxgb3_debug-mod \ --with-nes-mod \ --without-nes_debug-mod \ --without-memtrack \ --with-debug-info # make # make install On Thu, 2008-07-31 at 02:55 -0600, Harold/Carlyn Iuzzolino wrote: > Dear Openfabrics, (in particular Ralph Campbell ) > general at lists.openfabrics.org > > On Mon, 28 Jul 2008, Ralph Campbell wrote: > > > After looking at this somewhat, I don't see what the bug is. > > The various OFED-1.4 files in: > > kernel_addons/backport/*/include/linux/dma-mapping.h > > either have or don't have #ifndef __x86_64__ > > depending on whether valid_dma_direction() is defined in > > include/asm-x86_64/dma-mapping.h or not. > > For the kernel-devel-2.6.18-1.2798.fc6.x86_64.rpm, > > include/asm-x86_64/dma-mapping.h does define valid_dma_direction() > > and OFED-1.4 > > kernel_addons/backport/backport/2.6.18_FC6/include/linux/dma-mapping.h > > does correctly ifndef __x86_64__ valid_dma_direction(). > > > > I don't have an FC6 system handy so I can't reproduce this. > > > > Are you sure that you aren't installing on a patched kernel? > > You can check by looking at: > > cd /lib/modules/`uname -r`/build/include > > grep valid_dma_direction asm-x86_64/dma-mapping.h linux/dma-mapping.h > > > I'm using a clean Fedora 6 with NO UPDATES. I had to install the kernel-devel...rpm as well as some other rpms before I ran the OFED install.pl > > [root at elrond ~]# ls /lib/modules/ > 2.6.18-1.2798.fc6xen <--------- Only one directory > > [root at elrond ~]# uname -a > Linux elrond 2.6.18-1.2798.fc6xen #1 SMP Mon Oct 16 14:59:01 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux > [root at elrond ~]# uname -r > 2.6.18-1.2798.fc6xen > > [root at elrond include]# cd /lib/modules/`uname -r`/build/include > [root at elrond include]# pwd > /lib/modules/2.6.18-1.2798.fc6xen/build/include <----------- that's correct > > [root at elrond include]# ll > lrwxrwxrwx 1 root root 10 Jul 2 08:48 asm -> asm-x86_64 > drwxr-xr-x 3 root root 4096 Jul 2 08:48 asm-generic > drwxr-xr-x 11 root root 4096 Jul 2 08:48 asm-i386 > drwxr-xr-x 3 root root 4096 Jul 2 08:48 asm-x86_64 > > [root at elrond include]# more asm-x86_64/dma-mapping.h > > You are correct that the file asm-x86_64/dma-mapping.h DOES have a definition > of valid_dma_direction(int dma_direction) and it ISN'T ifndef'd out. > > > > > ------------------------------------- > > However, here is the compile command from /tmp/OFED.4905.logs/ofa_kernel.rpmbuild.log > of the subroutine ipath_dma.c, the one that has the complaint. > > gcc -Wp,-MD,/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/.ipath_dma.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.1.1/include -Iinclude/asm/mach-xen -D__KERNEL__ \ > -include include/linux/autoconf.h \ > -include /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/include/linux/autoconf.h \ > -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/ \ <--------------------------------- PROBLEM IS HERE > \ > \ > -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/include \ > -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/debug \ > -I/usr/local/include/scst \ > -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/ulp/srpt \ > -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/net/cxgb3 \ > -Iinclude \ > \ -D__XEN_INTERFACE_VERSION__=0x00030203 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit-function-declaration -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -fomit-frame-pointer -g > -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -DIPATH_IDSTR='"QLogic kernel.org driver"' -DIPATH_KERN_TYPE=0 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ipath_dma)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipath)" -c -o /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/.tmp_ipath_dma.o /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c > > Notice that there is no -I/lib/modules/2.6.18-1.2798.fc6xen/build/include > in the above gcc statement. According to what you said, that directory > has the correct dma-mapping.h file. The one that's being used, (see below) > has the static inline int valid_dma_direction(int dma_direction) code > #ifndef __x86_64__ around it. > > Now, here's the complaint > > /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c: In function 'ipath_dma_map_single': > /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.c:58: error: implicit declaration of function 'valid_dma_direction' > make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/hw/ipath/ipath_dma.o] Error 1 > > > > I think the problem is in this include directory > > -I/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/ \ <--------------------------------- PROBLEM IS HERE > > > [root at elrond include]# ll /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/ > total 28 > drwxr-xr-x 7 root root 4096 Jul 24 07:10 . > drwxr-xr-x 3 root root 4096 Jul 24 07:10 .. > drwxr-xr-x 2 root root 4096 Jul 24 07:10 asm > drwxr-xr-x 3 root root 4096 Jul 24 07:10 linux > drwxr-xr-x 2 root root 4096 Jul 24 07:10 net > drwxr-xr-x 2 root root 4096 Jul 24 07:10 scsi > drwxr-xr-x 2 root root 4096 Jul 24 07:10 src > > > ----------------- > > [root at elrond include]# ll /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h > -rw-r--r-- 1 root root 1076 Jul 24 07:10 /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h > > ----------------- > > [root at elrond include]# more /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h > > #include_next > > #ifndef __x86_64__ > > /* x86_64 was the only architecture that had valid_dma_direction > * before 2.6.18 */ > static inline int valid_dma_direction(int dma_direction) > { > return ((dma_direction == DMA_BIDIRECTIONAL) || > (dma_direction == DMA_TO_DEVICE) || > (dma_direction == DMA_FROM_DEVICE)); > } > #endif > > #endif > > > COMMENT: I don't think the directory /lib/modules/2.6.18-1.2798.fc6xen/build/include > or the file /lib/modules/2.6.18-1.2798.fc6xen/build/include/asm-x86_64/dma-mapping.h or > /lib/modules/2.6.18-1.2798.fc6xen/build/include/linux/dma-mapping.h > > is being used at all. > > Instead, the file /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18_FC6/include/linux/dma-mapping.h > is being used and it DOES have the definition #ifndef __x86_64__'d out > because this IS a 64 bit machine. > > > > > Now, this is a clean install of Fedora 6 and NOTHING has been updated, on > purpose. I used the install.pl script as is. So there's something in that > install script that makes it use the wrong directory for the include files. > > > Carlyn > > <<< Q: How do you make one burn? > <<< A: Differentiate a log fire. > <<< > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sfr at canb.auug.org.au Thu Jul 31 22:20:30 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Fri, 1 Aug 2008 15:20:30 +1000 Subject: [ofa-general] [PATCH v2] powerpc: move include files to arch/powerpc/include/asm Message-ID: <20080801152030.ff10b6b2.sfr@canb.auug.org.au> from include/asm-powerpc. This is the result of a mkdir arch/powerpc/include/asm git mv include/asm-powerpc/* arch/powerpc/include/asm Followed by a few documentation/comment fixups and a couple of places where was being used explicitly. Of the latter only one was outside the arch code and it is a driver only built for powerpc. Signed-off-by: Stephen Rothwell --- v2 don't change other arch files - the fixups are only in comments anyway. This patch can be applied with "git am" - the full patch is way to bug for our mailing lists. This has been built for all the powerpc defconfigs including all{no,mod,yes}config. There was only one failure, but that is expected anyway (I had to apply patches for the iommu and hfcmulti breakages). Documentation/powerpc/booting-without-of.txt | 4 ++-- Documentation/powerpc/eeh-pci-error-recovery.txt | 2 +- arch/powerpc/boot/io.h | 2 +- .../powerpc/include/asm}/8253pit.h | 0 .../powerpc/include/asm}/8xx_immap.h | 0 .../powerpc/include/asm}/Kbuild | 0 .../powerpc/include/asm}/a.out.h | 0 .../powerpc/include/asm}/abs_addr.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/agp.h | 0 .../powerpc/include/asm}/asm-compat.h | 0 .../powerpc/include/asm}/atomic.h | 0 .../powerpc/include/asm}/auxvec.h | 0 .../powerpc/include/asm}/backlight.h | 0 .../powerpc/include/asm}/bitops.h | 0 .../powerpc/include/asm}/bootx.h | 0 .../powerpc/include/asm}/btext.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/bug.h | 0 .../powerpc/include/asm}/bugs.h | 0 .../powerpc/include/asm}/byteorder.h | 0 .../powerpc/include/asm}/cache.h | 0 .../powerpc/include/asm}/cacheflush.h | 0 .../powerpc/include/asm}/cell-pmu.h | 0 .../powerpc/include/asm}/cell-regs.h | 0 .../powerpc/include/asm}/checksum.h | 0 .../powerpc/include/asm}/clk_interface.h | 0 .../powerpc/include/asm}/code-patching.h | 0 .../powerpc/include/asm}/compat.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/cpm.h | 0 .../powerpc/include/asm}/cpm1.h | 0 .../powerpc/include/asm}/cpm2.h | 0 .../powerpc/include/asm}/cputable.h | 0 .../powerpc/include/asm}/cputhreads.h | 0 .../powerpc/include/asm}/cputime.h | 0 .../powerpc/include/asm}/current.h | 0 .../powerpc/include/asm}/dbdma.h | 0 .../powerpc/include/asm}/dcr-generic.h | 0 .../powerpc/include/asm}/dcr-mmio.h | 0 .../powerpc/include/asm}/dcr-native.h | 0 .../powerpc/include/asm}/dcr-regs.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/dcr.h | 0 .../powerpc/include/asm}/delay.h | 0 .../powerpc/include/asm}/device.h | 0 .../powerpc/include/asm}/div64.h | 0 .../powerpc/include/asm}/dma-mapping.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/dma.h | 0 .../powerpc/include/asm}/edac.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/eeh.h | 0 .../powerpc/include/asm}/eeh_event.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/elf.h | 0 .../powerpc/include/asm}/emergency-restart.h | 0 .../powerpc/include/asm}/errno.h | 0 .../powerpc/include/asm}/exception.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/fb.h | 0 .../powerpc/include/asm}/fcntl.h | 0 .../powerpc/include/asm}/feature-fixups.h | 0 .../powerpc/include/asm}/firmware.h | 0 .../powerpc/include/asm}/fixmap.h | 0 .../powerpc/include/asm}/floppy.h | 0 .../powerpc/include/asm}/fs_pd.h | 0 .../powerpc/include/asm}/fsl_gtm.h | 0 .../powerpc/include/asm}/fsl_lbc.h | 0 .../powerpc/include/asm}/ftrace.h | 0 .../powerpc/include/asm}/futex.h | 0 .../powerpc/include/asm}/gpio.h | 0 .../powerpc/include/asm}/grackle.h | 0 .../powerpc/include/asm}/hardirq.h | 0 .../powerpc/include/asm}/heathrow.h | 0 .../powerpc/include/asm}/highmem.h | 0 .../powerpc/include/asm}/hugetlb.h | 0 .../powerpc/include/asm}/hvcall.h | 0 .../powerpc/include/asm}/hvconsole.h | 0 .../powerpc/include/asm}/hvcserver.h | 0 .../powerpc/include/asm}/hw_irq.h | 0 .../powerpc/include/asm}/hydra.h | 0 .../powerpc/include/asm}/i8259.h | 0 .../powerpc/include/asm}/ibmebus.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/ide.h | 0 .../powerpc/include/asm}/immap_86xx.h | 0 .../powerpc/include/asm}/immap_cpm2.h | 0 .../powerpc/include/asm}/immap_qe.h | 2 -- .../powerpc/include/asm}/io-defs.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/io.h | 0 .../powerpc/include/asm}/ioctl.h | 0 .../powerpc/include/asm}/ioctls.h | 0 .../powerpc/include/asm}/iommu.h | 0 .../powerpc/include/asm}/ipcbuf.h | 0 .../powerpc/include/asm}/ipic.h | 2 -- .../asm-powerpc => arch/powerpc/include/asm}/irq.h | 0 .../powerpc/include/asm}/irq_regs.h | 0 .../powerpc/include/asm}/irqflags.h | 4 +--- .../powerpc/include/asm}/iseries/alpaca.h | 0 .../powerpc/include/asm}/iseries/hv_call.h | 0 .../powerpc/include/asm}/iseries/hv_call_event.h | 0 .../powerpc/include/asm}/iseries/hv_call_sc.h | 0 .../powerpc/include/asm}/iseries/hv_call_xm.h | 0 .../powerpc/include/asm}/iseries/hv_lp_config.h | 0 .../powerpc/include/asm}/iseries/hv_lp_event.h | 0 .../powerpc/include/asm}/iseries/hv_types.h | 0 .../powerpc/include/asm}/iseries/iommu.h | 0 .../powerpc/include/asm}/iseries/it_lp_queue.h | 0 .../powerpc/include/asm}/iseries/lpar_map.h | 0 .../powerpc/include/asm}/iseries/mf.h | 0 .../powerpc/include/asm}/iseries/vio.h | 0 .../powerpc/include/asm}/kdebug.h | 0 .../powerpc/include/asm}/kdump.h | 0 .../powerpc/include/asm}/kexec.h | 0 .../powerpc/include/asm}/keylargo.h | 0 .../powerpc/include/asm}/kgdb.h | 2 -- .../powerpc/include/asm}/kmap_types.h | 0 .../powerpc/include/asm}/kprobes.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/kvm.h | 0 .../powerpc/include/asm}/kvm_asm.h | 0 .../powerpc/include/asm}/kvm_host.h | 0 .../powerpc/include/asm}/kvm_para.h | 0 .../powerpc/include/asm}/kvm_ppc.h | 0 .../powerpc/include/asm}/libata-portmap.h | 0 .../powerpc/include/asm}/linkage.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/lmb.h | 0 .../powerpc/include/asm}/local.h | 0 .../powerpc/include/asm}/lppaca.h | 0 .../powerpc/include/asm}/lv1call.h | 0 .../powerpc/include/asm}/machdep.h | 0 .../powerpc/include/asm}/macio.h | 0 .../powerpc/include/asm}/mc146818rtc.h | 0 .../powerpc/include/asm}/mediabay.h | 0 .../powerpc/include/asm}/mman.h | 0 .../powerpc/include/asm}/mmu-40x.h | 0 .../powerpc/include/asm}/mmu-44x.h | 0 .../powerpc/include/asm}/mmu-8xx.h | 0 .../powerpc/include/asm}/mmu-fsl-booke.h | 0 .../powerpc/include/asm}/mmu-hash32.h | 0 .../powerpc/include/asm}/mmu-hash64.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/mmu.h | 0 .../powerpc/include/asm}/mmu_context.h | 0 .../powerpc/include/asm}/mmzone.h | 0 .../powerpc/include/asm}/module.h | 0 .../powerpc/include/asm}/mpc512x.h | 0 .../powerpc/include/asm}/mpc52xx.h | 0 .../powerpc/include/asm}/mpc52xx_psc.h | 0 .../powerpc/include/asm}/mpc6xx.h | 0 .../powerpc/include/asm}/mpc8260.h | 0 .../powerpc/include/asm}/mpc86xx.h | 0 .../powerpc/include/asm}/mpc8xx.h | 0 .../powerpc/include/asm}/mpic.h | 0 .../powerpc/include/asm}/msgbuf.h | 0 .../powerpc/include/asm}/mutex.h | 0 .../powerpc/include/asm}/nvram.h | 0 .../powerpc/include/asm}/of_device.h | 0 .../powerpc/include/asm}/of_platform.h | 0 .../powerpc/include/asm}/ohare.h | 0 .../powerpc/include/asm}/oprofile_impl.h | 0 .../powerpc/include/asm}/pSeries_reconfig.h | 0 .../powerpc/include/asm}/paca.h | 2 -- .../powerpc/include/asm}/page.h | 0 .../powerpc/include/asm}/page_32.h | 0 .../powerpc/include/asm}/page_64.h | 0 .../powerpc/include/asm}/param.h | 0 .../powerpc/include/asm}/parport.h | 0 .../powerpc/include/asm}/pasemi_dma.h | 0 .../powerpc/include/asm}/pci-bridge.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/pci.h | 0 .../powerpc/include/asm}/percpu.h | 0 .../powerpc/include/asm}/pgalloc-32.h | 0 .../powerpc/include/asm}/pgalloc-64.h | 0 .../powerpc/include/asm}/pgalloc.h | 0 .../powerpc/include/asm}/pgtable-4k.h | 0 .../powerpc/include/asm}/pgtable-64k.h | 0 .../powerpc/include/asm}/pgtable-ppc32.h | 0 .../powerpc/include/asm}/pgtable-ppc64.h | 2 +- .../powerpc/include/asm}/pgtable.h | 0 .../powerpc/include/asm}/phyp_dump.h | 0 .../powerpc/include/asm}/pmac_feature.h | 0 .../powerpc/include/asm}/pmac_low_i2c.h | 0 .../powerpc/include/asm}/pmac_pfunc.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/pmc.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/pmi.h | 0 .../powerpc/include/asm}/poll.h | 0 .../powerpc/include/asm}/posix_types.h | 0 .../powerpc/include/asm}/ppc-pci.h | 0 .../powerpc/include/asm}/ppc4xx.h | 0 .../powerpc/include/asm}/ppc_asm.h | 0 .../powerpc/include/asm}/processor.h | 0 .../powerpc/include/asm}/prom.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/ps3.h | 0 .../powerpc/include/asm}/ps3av.h | 0 .../powerpc/include/asm}/ps3fb.h | 0 .../powerpc/include/asm}/ps3stor.h | 0 .../powerpc/include/asm}/ptrace.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/qe.h | 0 .../powerpc/include/asm}/qe_ic.h | 2 -- .../asm-powerpc => arch/powerpc/include/asm}/reg.h | 0 .../powerpc/include/asm}/reg_8xx.h | 0 .../powerpc/include/asm}/reg_booke.h | 0 .../powerpc/include/asm}/reg_fsl_emb.h | 0 .../powerpc/include/asm}/resource.h | 0 .../powerpc/include/asm}/rheap.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/rio.h | 0 .../powerpc/include/asm}/rtas.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/rtc.h | 0 .../powerpc/include/asm}/rwsem.h | 4 ++-- .../powerpc/include/asm}/scatterlist.h | 0 .../powerpc/include/asm}/seccomp.h | 0 .../powerpc/include/asm}/sections.h | 0 .../powerpc/include/asm}/sembuf.h | 0 .../powerpc/include/asm}/serial.h | 0 .../powerpc/include/asm}/setjmp.h | 0 .../powerpc/include/asm}/setup.h | 0 .../powerpc/include/asm}/shmbuf.h | 0 .../powerpc/include/asm}/shmparam.h | 0 .../powerpc/include/asm}/sigcontext.h | 0 .../powerpc/include/asm}/siginfo.h | 0 .../powerpc/include/asm}/signal.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/smp.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/smu.h | 0 .../powerpc/include/asm}/socket.h | 0 .../powerpc/include/asm}/sockios.h | 0 .../powerpc/include/asm}/sparsemem.h | 0 .../powerpc/include/asm}/spinlock.h | 0 .../powerpc/include/asm}/spinlock_types.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/spu.h | 0 .../powerpc/include/asm}/spu_csa.h | 0 .../powerpc/include/asm}/spu_info.h | 0 .../powerpc/include/asm}/spu_priv1.h | 0 .../powerpc/include/asm}/sstep.h | 0 .../powerpc/include/asm}/stat.h | 0 .../powerpc/include/asm}/statfs.h | 0 .../powerpc/include/asm}/string.h | 0 .../powerpc/include/asm}/suspend.h | 0 .../powerpc/include/asm}/synch.h | 0 .../powerpc/include/asm}/syscall.h | 0 .../powerpc/include/asm}/syscalls.h | 0 .../powerpc/include/asm}/systbl.h | 0 .../powerpc/include/asm}/system.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/tce.h | 0 .../powerpc/include/asm}/termbits.h | 0 .../powerpc/include/asm}/termios.h | 0 .../powerpc/include/asm}/thread_info.h | 0 .../powerpc/include/asm}/time.h | 0 .../powerpc/include/asm}/timex.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/tlb.h | 0 .../powerpc/include/asm}/tlbflush.h | 0 .../powerpc/include/asm}/topology.h | 0 .../powerpc/include/asm}/tsi108.h | 0 .../powerpc/include/asm}/tsi108_irq.h | 0 .../powerpc/include/asm}/tsi108_pci.h | 0 .../powerpc/include/asm}/types.h | 0 .../powerpc/include/asm}/uaccess.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/ucc.h | 0 .../powerpc/include/asm}/ucc_fast.h | 2 -- .../powerpc/include/asm}/ucc_slow.h | 0 .../powerpc/include/asm}/ucontext.h | 0 .../powerpc/include/asm}/udbg.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/uic.h | 2 -- .../powerpc/include/asm}/unaligned.h | 0 .../powerpc/include/asm}/uninorth.h | 0 .../powerpc/include/asm}/unistd.h | 0 .../powerpc/include/asm}/user.h | 0 .../powerpc/include/asm}/vdso.h | 0 .../powerpc/include/asm}/vdso_datapage.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/vga.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/vio.h | 0 .../powerpc/include/asm}/xilinx_intc.h | 0 .../powerpc/include/asm}/xmon.h | 0 .../asm-powerpc => arch/powerpc/include/asm}/xor.h | 0 arch/powerpc/mm/tlb_64.c | 2 +- arch/powerpc/platforms/86xx/mpc86xx_smp.c | 2 +- drivers/char/hvc_console.h | 2 +- drivers/char/hvcs.c | 2 +- drivers/infiniband/hw/ehca/ehca_reqs.c | 2 +- 269 files changed, 13 insertions(+), 29 deletions(-) rename {include/asm-powerpc => arch/powerpc/include/asm}/8253pit.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/8xx_immap.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/Kbuild (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/a.out.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/abs_addr.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/agp.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/asm-compat.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/atomic.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/auxvec.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/backlight.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/bitops.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/bootx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/btext.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/bug.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/bugs.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/byteorder.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cache.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cacheflush.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cell-pmu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cell-regs.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/checksum.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/clk_interface.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/code-patching.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/compat.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cpm.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cpm1.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cpm2.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cputable.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cputhreads.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/cputime.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/current.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dbdma.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dcr-generic.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dcr-mmio.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dcr-native.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dcr-regs.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dcr.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/delay.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/device.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/div64.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dma-mapping.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/dma.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/edac.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/eeh.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/eeh_event.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/elf.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/emergency-restart.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/errno.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/exception.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/fb.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/fcntl.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/feature-fixups.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/firmware.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/fixmap.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/floppy.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/fs_pd.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/fsl_gtm.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/fsl_lbc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ftrace.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/futex.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/gpio.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/grackle.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hardirq.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/heathrow.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/highmem.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hugetlb.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hvcall.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hvconsole.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hvcserver.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hw_irq.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/hydra.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/i8259.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ibmebus.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ide.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/immap_86xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/immap_cpm2.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/immap_qe.h (99%) rename {include/asm-powerpc => arch/powerpc/include/asm}/io-defs.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/io.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ioctl.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ioctls.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iommu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ipcbuf.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ipic.h (99%) rename {include/asm-powerpc => arch/powerpc/include/asm}/irq.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/irq_regs.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/irqflags.h (93%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/alpaca.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_call.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_call_event.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_call_sc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_call_xm.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_lp_config.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_lp_event.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/hv_types.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/iommu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/it_lp_queue.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/lpar_map.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/mf.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/iseries/vio.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kdebug.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kdump.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kexec.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/keylargo.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kgdb.h (98%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kmap_types.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kprobes.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kvm.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kvm_asm.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kvm_host.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kvm_para.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/kvm_ppc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/libata-portmap.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/linkage.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/lmb.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/local.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/lppaca.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/lv1call.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/machdep.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/macio.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mc146818rtc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mediabay.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mman.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu-40x.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu-44x.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu-8xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu-fsl-booke.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu-hash32.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu-hash64.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmu_context.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mmzone.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/module.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc512x.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc52xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc52xx_psc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc6xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc8260.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc86xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpc8xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mpic.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/msgbuf.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/mutex.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/nvram.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/of_device.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/of_platform.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ohare.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/oprofile_impl.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pSeries_reconfig.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/paca.h (99%) rename {include/asm-powerpc => arch/powerpc/include/asm}/page.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/page_32.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/page_64.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/param.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/parport.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pasemi_dma.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pci-bridge.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pci.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/percpu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgalloc-32.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgalloc-64.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgalloc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgtable-4k.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgtable-64k.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgtable-ppc32.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgtable-ppc64.h (99%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pgtable.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/phyp_dump.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pmac_feature.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pmac_low_i2c.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pmac_pfunc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pmc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/pmi.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/poll.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/posix_types.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ppc-pci.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ppc4xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ppc_asm.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/processor.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/prom.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ps3.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ps3av.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ps3fb.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ps3stor.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ptrace.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/qe.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/qe_ic.h (99%) rename {include/asm-powerpc => arch/powerpc/include/asm}/reg.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/reg_8xx.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/reg_booke.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/reg_fsl_emb.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/resource.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/rheap.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/rio.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/rtas.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/rtc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/rwsem.h (96%) rename {include/asm-powerpc => arch/powerpc/include/asm}/scatterlist.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/seccomp.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/sections.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/sembuf.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/serial.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/setjmp.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/setup.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/shmbuf.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/shmparam.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/sigcontext.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/siginfo.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/signal.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/smp.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/smu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/socket.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/sockios.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/sparsemem.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/spinlock.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/spinlock_types.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/spu.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/spu_csa.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/spu_info.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/spu_priv1.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/sstep.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/stat.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/statfs.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/string.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/suspend.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/synch.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/syscall.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/syscalls.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/systbl.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/system.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/tce.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/termbits.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/termios.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/thread_info.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/time.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/timex.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/tlb.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/tlbflush.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/topology.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/tsi108.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/tsi108_irq.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/tsi108_pci.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/types.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/uaccess.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ucc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ucc_fast.h (99%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ucc_slow.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/ucontext.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/udbg.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/uic.h (95%) rename {include/asm-powerpc => arch/powerpc/include/asm}/unaligned.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/uninorth.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/unistd.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/user.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/vdso.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/vdso_datapage.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/vga.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/vio.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/xilinx_intc.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/xmon.h (100%) rename {include/asm-powerpc => arch/powerpc/include/asm}/xor.h (100%) diff --git a/Documentation/powerpc/booting-without-of.txt b/Documentation/powerpc/booting-without-of.txt index 928a79c..de4063c 100644 --- a/Documentation/powerpc/booting-without-of.txt +++ b/Documentation/powerpc/booting-without-of.txt @@ -278,7 +278,7 @@ it with special cases. a 64-bit platform. d) request and get assigned a platform number (see PLATFORM_* - constants in include/asm-powerpc/processor.h + constants in arch/powerpc/include/asm/processor.h 32-bit embedded kernels: @@ -340,7 +340,7 @@ the block to RAM before passing it to the kernel. --------- The kernel is entered with r3 pointing to an area of memory that is - roughly described in include/asm-powerpc/prom.h by the structure + roughly described in arch/powerpc/include/asm/prom.h by the structure boot_param_header: struct boot_param_header { diff --git a/Documentation/powerpc/eeh-pci-error-recovery.txt b/Documentation/powerpc/eeh-pci-error-recovery.txt index df7afe4..9d4e33d 100644 --- a/Documentation/powerpc/eeh-pci-error-recovery.txt +++ b/Documentation/powerpc/eeh-pci-error-recovery.txt @@ -133,7 +133,7 @@ error. Given an arbitrary address, the routine pci_get_device_by_addr() will find the pci device associated with that address (if any). -The default include/asm-powerpc/io.h macros readb(), inb(), insb(), +The default arch/powerpc/include/asm/io.h macros readb(), inb(), insb(), etc. include a check to see if the i/o read returned all-0xff's. If so, these make a call to eeh_dn_check_failure(), which in turn asks the firmware if the all-ff's value is the sign of a true EEH diff --git a/arch/powerpc/boot/io.h b/arch/powerpc/boot/io.h index ccaedae..7c09f48 100644 --- a/arch/powerpc/boot/io.h +++ b/arch/powerpc/boot/io.h @@ -6,7 +6,7 @@ /* * Low-level I/O routines. * - * Copied from (which has no copyright) + * Copied from (which has no copyright) */ static inline int in_8(const volatile unsigned char *addr) { diff --git a/include/asm-powerpc/8253pit.h b/arch/powerpc/include/asm/8253pit.h similarity index 100% rename from include/asm-powerpc/8253pit.h rename to arch/powerpc/include/asm/8253pit.h diff --git a/include/asm-powerpc/8xx_immap.h b/arch/powerpc/include/asm/8xx_immap.h similarity index 100% rename from include/asm-powerpc/8xx_immap.h rename to arch/powerpc/include/asm/8xx_immap.h diff --git a/include/asm-powerpc/Kbuild b/arch/powerpc/include/asm/Kbuild similarity index 100% rename from include/asm-powerpc/Kbuild rename to arch/powerpc/include/asm/Kbuild diff --git a/include/asm-powerpc/a.out.h b/arch/powerpc/include/asm/a.out.h similarity index 100% rename from include/asm-powerpc/a.out.h rename to arch/powerpc/include/asm/a.out.h diff --git a/include/asm-powerpc/abs_addr.h b/arch/powerpc/include/asm/abs_addr.h similarity index 100% rename from include/asm-powerpc/abs_addr.h rename to arch/powerpc/include/asm/abs_addr.h diff --git a/include/asm-powerpc/agp.h b/arch/powerpc/include/asm/agp.h similarity index 100% rename from include/asm-powerpc/agp.h rename to arch/powerpc/include/asm/agp.h diff --git a/include/asm-powerpc/asm-compat.h b/arch/powerpc/include/asm/asm-compat.h similarity index 100% rename from include/asm-powerpc/asm-compat.h rename to arch/powerpc/include/asm/asm-compat.h diff --git a/include/asm-powerpc/atomic.h b/arch/powerpc/include/asm/atomic.h similarity index 100% rename from include/asm-powerpc/atomic.h rename to arch/powerpc/include/asm/atomic.h diff --git a/include/asm-powerpc/auxvec.h b/arch/powerpc/include/asm/auxvec.h similarity index 100% rename from include/asm-powerpc/auxvec.h rename to arch/powerpc/include/asm/auxvec.h diff --git a/include/asm-powerpc/backlight.h b/arch/powerpc/include/asm/backlight.h similarity index 100% rename from include/asm-powerpc/backlight.h rename to arch/powerpc/include/asm/backlight.h diff --git a/include/asm-powerpc/bitops.h b/arch/powerpc/include/asm/bitops.h similarity index 100% rename from include/asm-powerpc/bitops.h rename to arch/powerpc/include/asm/bitops.h diff --git a/include/asm-powerpc/bootx.h b/arch/powerpc/include/asm/bootx.h similarity index 100% rename from include/asm-powerpc/bootx.h rename to arch/powerpc/include/asm/bootx.h diff --git a/include/asm-powerpc/btext.h b/arch/powerpc/include/asm/btext.h similarity index 100% rename from include/asm-powerpc/btext.h rename to arch/powerpc/include/asm/btext.h diff --git a/include/asm-powerpc/bug.h b/arch/powerpc/include/asm/bug.h similarity index 100% rename from include/asm-powerpc/bug.h rename to arch/powerpc/include/asm/bug.h diff --git a/include/asm-powerpc/bugs.h b/arch/powerpc/include/asm/bugs.h similarity index 100% rename from include/asm-powerpc/bugs.h rename to arch/powerpc/include/asm/bugs.h diff --git a/include/asm-powerpc/byteorder.h b/arch/powerpc/include/asm/byteorder.h similarity index 100% rename from include/asm-powerpc/byteorder.h rename to arch/powerpc/include/asm/byteorder.h diff --git a/include/asm-powerpc/cache.h b/arch/powerpc/include/asm/cache.h similarity index 100% rename from include/asm-powerpc/cache.h rename to arch/powerpc/include/asm/cache.h diff --git a/include/asm-powerpc/cacheflush.h b/arch/powerpc/include/asm/cacheflush.h similarity index 100% rename from include/asm-powerpc/cacheflush.h rename to arch/powerpc/include/asm/cacheflush.h diff --git a/include/asm-powerpc/cell-pmu.h b/arch/powerpc/include/asm/cell-pmu.h similarity index 100% rename from include/asm-powerpc/cell-pmu.h rename to arch/powerpc/include/asm/cell-pmu.h diff --git a/include/asm-powerpc/cell-regs.h b/arch/powerpc/include/asm/cell-regs.h similarity index 100% rename from include/asm-powerpc/cell-regs.h rename to arch/powerpc/include/asm/cell-regs.h diff --git a/include/asm-powerpc/checksum.h b/arch/powerpc/include/asm/checksum.h similarity index 100% rename from include/asm-powerpc/checksum.h rename to arch/powerpc/include/asm/checksum.h diff --git a/include/asm-powerpc/clk_interface.h b/arch/powerpc/include/asm/clk_interface.h similarity index 100% rename from include/asm-powerpc/clk_interface.h rename to arch/powerpc/include/asm/clk_interface.h diff --git a/include/asm-powerpc/code-patching.h b/arch/powerpc/include/asm/code-patching.h similarity index 100% rename from include/asm-powerpc/code-patching.h rename to arch/powerpc/include/asm/code-patching.h diff --git a/include/asm-powerpc/compat.h b/arch/powerpc/include/asm/compat.h similarity index 100% rename from include/asm-powerpc/compat.h rename to arch/powerpc/include/asm/compat.h diff --git a/include/asm-powerpc/cpm.h b/arch/powerpc/include/asm/cpm.h similarity index 100% rename from include/asm-powerpc/cpm.h rename to arch/powerpc/include/asm/cpm.h diff --git a/include/asm-powerpc/cpm1.h b/arch/powerpc/include/asm/cpm1.h similarity index 100% rename from include/asm-powerpc/cpm1.h rename to arch/powerpc/include/asm/cpm1.h diff --git a/include/asm-powerpc/cpm2.h b/arch/powerpc/include/asm/cpm2.h similarity index 100% rename from include/asm-powerpc/cpm2.h rename to arch/powerpc/include/asm/cpm2.h diff --git a/include/asm-powerpc/cputable.h b/arch/powerpc/include/asm/cputable.h similarity index 100% rename from include/asm-powerpc/cputable.h rename to arch/powerpc/include/asm/cputable.h diff --git a/include/asm-powerpc/cputhreads.h b/arch/powerpc/include/asm/cputhreads.h similarity index 100% rename from include/asm-powerpc/cputhreads.h rename to arch/powerpc/include/asm/cputhreads.h diff --git a/include/asm-powerpc/cputime.h b/arch/powerpc/include/asm/cputime.h similarity index 100% rename from include/asm-powerpc/cputime.h rename to arch/powerpc/include/asm/cputime.h diff --git a/include/asm-powerpc/current.h b/arch/powerpc/include/asm/current.h similarity index 100% rename from include/asm-powerpc/current.h rename to arch/powerpc/include/asm/current.h diff --git a/include/asm-powerpc/dbdma.h b/arch/powerpc/include/asm/dbdma.h similarity index 100% rename from include/asm-powerpc/dbdma.h rename to arch/powerpc/include/asm/dbdma.h diff --git a/include/asm-powerpc/dcr-generic.h b/arch/powerpc/include/asm/dcr-generic.h similarity index 100% rename from include/asm-powerpc/dcr-generic.h rename to arch/powerpc/include/asm/dcr-generic.h diff --git a/include/asm-powerpc/dcr-mmio.h b/arch/powerpc/include/asm/dcr-mmio.h similarity index 100% rename from include/asm-powerpc/dcr-mmio.h rename to arch/powerpc/include/asm/dcr-mmio.h diff --git a/include/asm-powerpc/dcr-native.h b/arch/powerpc/include/asm/dcr-native.h similarity index 100% rename from include/asm-powerpc/dcr-native.h rename to arch/powerpc/include/asm/dcr-native.h diff --git a/include/asm-powerpc/dcr-regs.h b/arch/powerpc/include/asm/dcr-regs.h similarity index 100% rename from include/asm-powerpc/dcr-regs.h rename to arch/powerpc/include/asm/dcr-regs.h diff --git a/include/asm-powerpc/dcr.h b/arch/powerpc/include/asm/dcr.h similarity index 100% rename from include/asm-powerpc/dcr.h rename to arch/powerpc/include/asm/dcr.h diff --git a/include/asm-powerpc/delay.h b/arch/powerpc/include/asm/delay.h similarity index 100% rename from include/asm-powerpc/delay.h rename to arch/powerpc/include/asm/delay.h diff --git a/include/asm-powerpc/device.h b/arch/powerpc/include/asm/device.h similarity index 100% rename from include/asm-powerpc/device.h rename to arch/powerpc/include/asm/device.h diff --git a/include/asm-powerpc/div64.h b/arch/powerpc/include/asm/div64.h similarity index 100% rename from include/asm-powerpc/div64.h rename to arch/powerpc/include/asm/div64.h diff --git a/include/asm-powerpc/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h similarity index 100% rename from include/asm-powerpc/dma-mapping.h rename to arch/powerpc/include/asm/dma-mapping.h diff --git a/include/asm-powerpc/dma.h b/arch/powerpc/include/asm/dma.h similarity index 100% rename from include/asm-powerpc/dma.h rename to arch/powerpc/include/asm/dma.h diff --git a/include/asm-powerpc/edac.h b/arch/powerpc/include/asm/edac.h similarity index 100% rename from include/asm-powerpc/edac.h rename to arch/powerpc/include/asm/edac.h diff --git a/include/asm-powerpc/eeh.h b/arch/powerpc/include/asm/eeh.h similarity index 100% rename from include/asm-powerpc/eeh.h rename to arch/powerpc/include/asm/eeh.h diff --git a/include/asm-powerpc/eeh_event.h b/arch/powerpc/include/asm/eeh_event.h similarity index 100% rename from include/asm-powerpc/eeh_event.h rename to arch/powerpc/include/asm/eeh_event.h diff --git a/include/asm-powerpc/elf.h b/arch/powerpc/include/asm/elf.h similarity index 100% rename from include/asm-powerpc/elf.h rename to arch/powerpc/include/asm/elf.h diff --git a/include/asm-powerpc/emergency-restart.h b/arch/powerpc/include/asm/emergency-restart.h similarity index 100% rename from include/asm-powerpc/emergency-restart.h rename to arch/powerpc/include/asm/emergency-restart.h diff --git a/include/asm-powerpc/errno.h b/arch/powerpc/include/asm/errno.h similarity index 100% rename from include/asm-powerpc/errno.h rename to arch/powerpc/include/asm/errno.h diff --git a/include/asm-powerpc/exception.h b/arch/powerpc/include/asm/exception.h similarity index 100% rename from include/asm-powerpc/exception.h rename to arch/powerpc/include/asm/exception.h diff --git a/include/asm-powerpc/fb.h b/arch/powerpc/include/asm/fb.h similarity index 100% rename from include/asm-powerpc/fb.h rename to arch/powerpc/include/asm/fb.h diff --git a/include/asm-powerpc/fcntl.h b/arch/powerpc/include/asm/fcntl.h similarity index 100% rename from include/asm-powerpc/fcntl.h rename to arch/powerpc/include/asm/fcntl.h diff --git a/include/asm-powerpc/feature-fixups.h b/arch/powerpc/include/asm/feature-fixups.h similarity index 100% rename from include/asm-powerpc/feature-fixups.h rename to arch/powerpc/include/asm/feature-fixups.h diff --git a/include/asm-powerpc/firmware.h b/arch/powerpc/include/asm/firmware.h similarity index 100% rename from include/asm-powerpc/firmware.h rename to arch/powerpc/include/asm/firmware.h diff --git a/include/asm-powerpc/fixmap.h b/arch/powerpc/include/asm/fixmap.h similarity index 100% rename from include/asm-powerpc/fixmap.h rename to arch/powerpc/include/asm/fixmap.h diff --git a/include/asm-powerpc/floppy.h b/arch/powerpc/include/asm/floppy.h similarity index 100% rename from include/asm-powerpc/floppy.h rename to arch/powerpc/include/asm/floppy.h diff --git a/include/asm-powerpc/fs_pd.h b/arch/powerpc/include/asm/fs_pd.h similarity index 100% rename from include/asm-powerpc/fs_pd.h rename to arch/powerpc/include/asm/fs_pd.h diff --git a/include/asm-powerpc/fsl_gtm.h b/arch/powerpc/include/asm/fsl_gtm.h similarity index 100% rename from include/asm-powerpc/fsl_gtm.h rename to arch/powerpc/include/asm/fsl_gtm.h diff --git a/include/asm-powerpc/fsl_lbc.h b/arch/powerpc/include/asm/fsl_lbc.h similarity index 100% rename from include/asm-powerpc/fsl_lbc.h rename to arch/powerpc/include/asm/fsl_lbc.h diff --git a/include/asm-powerpc/ftrace.h b/arch/powerpc/include/asm/ftrace.h similarity index 100% rename from include/asm-powerpc/ftrace.h rename to arch/powerpc/include/asm/ftrace.h diff --git a/include/asm-powerpc/futex.h b/arch/powerpc/include/asm/futex.h similarity index 100% rename from include/asm-powerpc/futex.h rename to arch/powerpc/include/asm/futex.h diff --git a/include/asm-powerpc/gpio.h b/arch/powerpc/include/asm/gpio.h similarity index 100% rename from include/asm-powerpc/gpio.h rename to arch/powerpc/include/asm/gpio.h diff --git a/include/asm-powerpc/grackle.h b/arch/powerpc/include/asm/grackle.h similarity index 100% rename from include/asm-powerpc/grackle.h rename to arch/powerpc/include/asm/grackle.h diff --git a/include/asm-powerpc/hardirq.h b/arch/powerpc/include/asm/hardirq.h similarity index 100% rename from include/asm-powerpc/hardirq.h rename to arch/powerpc/include/asm/hardirq.h diff --git a/include/asm-powerpc/heathrow.h b/arch/powerpc/include/asm/heathrow.h similarity index 100% rename from include/asm-powerpc/heathrow.h rename to arch/powerpc/include/asm/heathrow.h diff --git a/include/asm-powerpc/highmem.h b/arch/powerpc/include/asm/highmem.h similarity index 100% rename from include/asm-powerpc/highmem.h rename to arch/powerpc/include/asm/highmem.h diff --git a/include/asm-powerpc/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h similarity index 100% rename from include/asm-powerpc/hugetlb.h rename to arch/powerpc/include/asm/hugetlb.h diff --git a/include/asm-powerpc/hvcall.h b/arch/powerpc/include/asm/hvcall.h similarity index 100% rename from include/asm-powerpc/hvcall.h rename to arch/powerpc/include/asm/hvcall.h diff --git a/include/asm-powerpc/hvconsole.h b/arch/powerpc/include/asm/hvconsole.h similarity index 100% rename from include/asm-powerpc/hvconsole.h rename to arch/powerpc/include/asm/hvconsole.h diff --git a/include/asm-powerpc/hvcserver.h b/arch/powerpc/include/asm/hvcserver.h similarity index 100% rename from include/asm-powerpc/hvcserver.h rename to arch/powerpc/include/asm/hvcserver.h diff --git a/include/asm-powerpc/hw_irq.h b/arch/powerpc/include/asm/hw_irq.h similarity index 100% rename from include/asm-powerpc/hw_irq.h rename to arch/powerpc/include/asm/hw_irq.h diff --git a/include/asm-powerpc/hydra.h b/arch/powerpc/include/asm/hydra.h similarity index 100% rename from include/asm-powerpc/hydra.h rename to arch/powerpc/include/asm/hydra.h diff --git a/include/asm-powerpc/i8259.h b/arch/powerpc/include/asm/i8259.h similarity index 100% rename from include/asm-powerpc/i8259.h rename to arch/powerpc/include/asm/i8259.h diff --git a/include/asm-powerpc/ibmebus.h b/arch/powerpc/include/asm/ibmebus.h similarity index 100% rename from include/asm-powerpc/ibmebus.h rename to arch/powerpc/include/asm/ibmebus.h diff --git a/include/asm-powerpc/ide.h b/arch/powerpc/include/asm/ide.h similarity index 100% rename from include/asm-powerpc/ide.h rename to arch/powerpc/include/asm/ide.h diff --git a/include/asm-powerpc/immap_86xx.h b/arch/powerpc/include/asm/immap_86xx.h similarity index 100% rename from include/asm-powerpc/immap_86xx.h rename to arch/powerpc/include/asm/immap_86xx.h diff --git a/include/asm-powerpc/immap_cpm2.h b/arch/powerpc/include/asm/immap_cpm2.h similarity index 100% rename from include/asm-powerpc/immap_cpm2.h rename to arch/powerpc/include/asm/immap_cpm2.h diff --git a/include/asm-powerpc/immap_qe.h b/arch/powerpc/include/asm/immap_qe.h similarity index 99% rename from include/asm-powerpc/immap_qe.h rename to arch/powerpc/include/asm/immap_qe.h index 7b6f411..3c2fced 100644 --- a/include/asm-powerpc/immap_qe.h +++ b/arch/powerpc/include/asm/immap_qe.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/immap_qe.h - * * QUICC Engine (QE) Internal Memory Map. * The Internal Memory Map for devices with QE on them. This * is the superset of all QE devices (8360, etc.). diff --git a/include/asm-powerpc/io-defs.h b/arch/powerpc/include/asm/io-defs.h similarity index 100% rename from include/asm-powerpc/io-defs.h rename to arch/powerpc/include/asm/io-defs.h diff --git a/include/asm-powerpc/io.h b/arch/powerpc/include/asm/io.h similarity index 100% rename from include/asm-powerpc/io.h rename to arch/powerpc/include/asm/io.h diff --git a/include/asm-powerpc/ioctl.h b/arch/powerpc/include/asm/ioctl.h similarity index 100% rename from include/asm-powerpc/ioctl.h rename to arch/powerpc/include/asm/ioctl.h diff --git a/include/asm-powerpc/ioctls.h b/arch/powerpc/include/asm/ioctls.h similarity index 100% rename from include/asm-powerpc/ioctls.h rename to arch/powerpc/include/asm/ioctls.h diff --git a/include/asm-powerpc/iommu.h b/arch/powerpc/include/asm/iommu.h similarity index 100% rename from include/asm-powerpc/iommu.h rename to arch/powerpc/include/asm/iommu.h diff --git a/include/asm-powerpc/ipcbuf.h b/arch/powerpc/include/asm/ipcbuf.h similarity index 100% rename from include/asm-powerpc/ipcbuf.h rename to arch/powerpc/include/asm/ipcbuf.h diff --git a/include/asm-powerpc/ipic.h b/arch/powerpc/include/asm/ipic.h similarity index 99% rename from include/asm-powerpc/ipic.h rename to arch/powerpc/include/asm/ipic.h index 8ff08be..4cf3553 100644 --- a/include/asm-powerpc/ipic.h +++ b/arch/powerpc/include/asm/ipic.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/ipic.h - * * IPIC external definitions and structure. * * Maintainer: Kumar Gala diff --git a/include/asm-powerpc/irq.h b/arch/powerpc/include/asm/irq.h similarity index 100% rename from include/asm-powerpc/irq.h rename to arch/powerpc/include/asm/irq.h diff --git a/include/asm-powerpc/irq_regs.h b/arch/powerpc/include/asm/irq_regs.h similarity index 100% rename from include/asm-powerpc/irq_regs.h rename to arch/powerpc/include/asm/irq_regs.h diff --git a/include/asm-powerpc/irqflags.h b/arch/powerpc/include/asm/irqflags.h similarity index 93% rename from include/asm-powerpc/irqflags.h rename to arch/powerpc/include/asm/irqflags.h index cc6fdba..17ba3a8 100644 --- a/include/asm-powerpc/irqflags.h +++ b/arch/powerpc/include/asm/irqflags.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/irqflags.h - * * IRQ flags handling */ #ifndef _ASM_IRQFLAGS_H @@ -10,7 +8,7 @@ /* * Get definitions for raw_local_save_flags(x), etc. */ -#include +#include #else #ifdef CONFIG_TRACE_IRQFLAGS diff --git a/include/asm-powerpc/iseries/alpaca.h b/arch/powerpc/include/asm/iseries/alpaca.h similarity index 100% rename from include/asm-powerpc/iseries/alpaca.h rename to arch/powerpc/include/asm/iseries/alpaca.h diff --git a/include/asm-powerpc/iseries/hv_call.h b/arch/powerpc/include/asm/iseries/hv_call.h similarity index 100% rename from include/asm-powerpc/iseries/hv_call.h rename to arch/powerpc/include/asm/iseries/hv_call.h diff --git a/include/asm-powerpc/iseries/hv_call_event.h b/arch/powerpc/include/asm/iseries/hv_call_event.h similarity index 100% rename from include/asm-powerpc/iseries/hv_call_event.h rename to arch/powerpc/include/asm/iseries/hv_call_event.h diff --git a/include/asm-powerpc/iseries/hv_call_sc.h b/arch/powerpc/include/asm/iseries/hv_call_sc.h similarity index 100% rename from include/asm-powerpc/iseries/hv_call_sc.h rename to arch/powerpc/include/asm/iseries/hv_call_sc.h diff --git a/include/asm-powerpc/iseries/hv_call_xm.h b/arch/powerpc/include/asm/iseries/hv_call_xm.h similarity index 100% rename from include/asm-powerpc/iseries/hv_call_xm.h rename to arch/powerpc/include/asm/iseries/hv_call_xm.h diff --git a/include/asm-powerpc/iseries/hv_lp_config.h b/arch/powerpc/include/asm/iseries/hv_lp_config.h similarity index 100% rename from include/asm-powerpc/iseries/hv_lp_config.h rename to arch/powerpc/include/asm/iseries/hv_lp_config.h diff --git a/include/asm-powerpc/iseries/hv_lp_event.h b/arch/powerpc/include/asm/iseries/hv_lp_event.h similarity index 100% rename from include/asm-powerpc/iseries/hv_lp_event.h rename to arch/powerpc/include/asm/iseries/hv_lp_event.h diff --git a/include/asm-powerpc/iseries/hv_types.h b/arch/powerpc/include/asm/iseries/hv_types.h similarity index 100% rename from include/asm-powerpc/iseries/hv_types.h rename to arch/powerpc/include/asm/iseries/hv_types.h diff --git a/include/asm-powerpc/iseries/iommu.h b/arch/powerpc/include/asm/iseries/iommu.h similarity index 100% rename from include/asm-powerpc/iseries/iommu.h rename to arch/powerpc/include/asm/iseries/iommu.h diff --git a/include/asm-powerpc/iseries/it_lp_queue.h b/arch/powerpc/include/asm/iseries/it_lp_queue.h similarity index 100% rename from include/asm-powerpc/iseries/it_lp_queue.h rename to arch/powerpc/include/asm/iseries/it_lp_queue.h diff --git a/include/asm-powerpc/iseries/lpar_map.h b/arch/powerpc/include/asm/iseries/lpar_map.h similarity index 100% rename from include/asm-powerpc/iseries/lpar_map.h rename to arch/powerpc/include/asm/iseries/lpar_map.h diff --git a/include/asm-powerpc/iseries/mf.h b/arch/powerpc/include/asm/iseries/mf.h similarity index 100% rename from include/asm-powerpc/iseries/mf.h rename to arch/powerpc/include/asm/iseries/mf.h diff --git a/include/asm-powerpc/iseries/vio.h b/arch/powerpc/include/asm/iseries/vio.h similarity index 100% rename from include/asm-powerpc/iseries/vio.h rename to arch/powerpc/include/asm/iseries/vio.h diff --git a/include/asm-powerpc/kdebug.h b/arch/powerpc/include/asm/kdebug.h similarity index 100% rename from include/asm-powerpc/kdebug.h rename to arch/powerpc/include/asm/kdebug.h diff --git a/include/asm-powerpc/kdump.h b/arch/powerpc/include/asm/kdump.h similarity index 100% rename from include/asm-powerpc/kdump.h rename to arch/powerpc/include/asm/kdump.h diff --git a/include/asm-powerpc/kexec.h b/arch/powerpc/include/asm/kexec.h similarity index 100% rename from include/asm-powerpc/kexec.h rename to arch/powerpc/include/asm/kexec.h diff --git a/include/asm-powerpc/keylargo.h b/arch/powerpc/include/asm/keylargo.h similarity index 100% rename from include/asm-powerpc/keylargo.h rename to arch/powerpc/include/asm/keylargo.h diff --git a/include/asm-powerpc/kgdb.h b/arch/powerpc/include/asm/kgdb.h similarity index 98% rename from include/asm-powerpc/kgdb.h rename to arch/powerpc/include/asm/kgdb.h index 1399caf..edd2170 100644 --- a/include/asm-powerpc/kgdb.h +++ b/arch/powerpc/include/asm/kgdb.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/kgdb.h - * * The PowerPC (32/64) specific defines / externs for KGDB. Based on * the previous 32bit and 64bit specific files, which had the following * copyrights: diff --git a/include/asm-powerpc/kmap_types.h b/arch/powerpc/include/asm/kmap_types.h similarity index 100% rename from include/asm-powerpc/kmap_types.h rename to arch/powerpc/include/asm/kmap_types.h diff --git a/include/asm-powerpc/kprobes.h b/arch/powerpc/include/asm/kprobes.h similarity index 100% rename from include/asm-powerpc/kprobes.h rename to arch/powerpc/include/asm/kprobes.h diff --git a/include/asm-powerpc/kvm.h b/arch/powerpc/include/asm/kvm.h similarity index 100% rename from include/asm-powerpc/kvm.h rename to arch/powerpc/include/asm/kvm.h diff --git a/include/asm-powerpc/kvm_asm.h b/arch/powerpc/include/asm/kvm_asm.h similarity index 100% rename from include/asm-powerpc/kvm_asm.h rename to arch/powerpc/include/asm/kvm_asm.h diff --git a/include/asm-powerpc/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h similarity index 100% rename from include/asm-powerpc/kvm_host.h rename to arch/powerpc/include/asm/kvm_host.h diff --git a/include/asm-powerpc/kvm_para.h b/arch/powerpc/include/asm/kvm_para.h similarity index 100% rename from include/asm-powerpc/kvm_para.h rename to arch/powerpc/include/asm/kvm_para.h diff --git a/include/asm-powerpc/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h similarity index 100% rename from include/asm-powerpc/kvm_ppc.h rename to arch/powerpc/include/asm/kvm_ppc.h diff --git a/include/asm-powerpc/libata-portmap.h b/arch/powerpc/include/asm/libata-portmap.h similarity index 100% rename from include/asm-powerpc/libata-portmap.h rename to arch/powerpc/include/asm/libata-portmap.h diff --git a/include/asm-powerpc/linkage.h b/arch/powerpc/include/asm/linkage.h similarity index 100% rename from include/asm-powerpc/linkage.h rename to arch/powerpc/include/asm/linkage.h diff --git a/include/asm-powerpc/lmb.h b/arch/powerpc/include/asm/lmb.h similarity index 100% rename from include/asm-powerpc/lmb.h rename to arch/powerpc/include/asm/lmb.h diff --git a/include/asm-powerpc/local.h b/arch/powerpc/include/asm/local.h similarity index 100% rename from include/asm-powerpc/local.h rename to arch/powerpc/include/asm/local.h diff --git a/include/asm-powerpc/lppaca.h b/arch/powerpc/include/asm/lppaca.h similarity index 100% rename from include/asm-powerpc/lppaca.h rename to arch/powerpc/include/asm/lppaca.h diff --git a/include/asm-powerpc/lv1call.h b/arch/powerpc/include/asm/lv1call.h similarity index 100% rename from include/asm-powerpc/lv1call.h rename to arch/powerpc/include/asm/lv1call.h diff --git a/include/asm-powerpc/machdep.h b/arch/powerpc/include/asm/machdep.h similarity index 100% rename from include/asm-powerpc/machdep.h rename to arch/powerpc/include/asm/machdep.h diff --git a/include/asm-powerpc/macio.h b/arch/powerpc/include/asm/macio.h similarity index 100% rename from include/asm-powerpc/macio.h rename to arch/powerpc/include/asm/macio.h diff --git a/include/asm-powerpc/mc146818rtc.h b/arch/powerpc/include/asm/mc146818rtc.h similarity index 100% rename from include/asm-powerpc/mc146818rtc.h rename to arch/powerpc/include/asm/mc146818rtc.h diff --git a/include/asm-powerpc/mediabay.h b/arch/powerpc/include/asm/mediabay.h similarity index 100% rename from include/asm-powerpc/mediabay.h rename to arch/powerpc/include/asm/mediabay.h diff --git a/include/asm-powerpc/mman.h b/arch/powerpc/include/asm/mman.h similarity index 100% rename from include/asm-powerpc/mman.h rename to arch/powerpc/include/asm/mman.h diff --git a/include/asm-powerpc/mmu-40x.h b/arch/powerpc/include/asm/mmu-40x.h similarity index 100% rename from include/asm-powerpc/mmu-40x.h rename to arch/powerpc/include/asm/mmu-40x.h diff --git a/include/asm-powerpc/mmu-44x.h b/arch/powerpc/include/asm/mmu-44x.h similarity index 100% rename from include/asm-powerpc/mmu-44x.h rename to arch/powerpc/include/asm/mmu-44x.h diff --git a/include/asm-powerpc/mmu-8xx.h b/arch/powerpc/include/asm/mmu-8xx.h similarity index 100% rename from include/asm-powerpc/mmu-8xx.h rename to arch/powerpc/include/asm/mmu-8xx.h diff --git a/include/asm-powerpc/mmu-fsl-booke.h b/arch/powerpc/include/asm/mmu-fsl-booke.h similarity index 100% rename from include/asm-powerpc/mmu-fsl-booke.h rename to arch/powerpc/include/asm/mmu-fsl-booke.h diff --git a/include/asm-powerpc/mmu-hash32.h b/arch/powerpc/include/asm/mmu-hash32.h similarity index 100% rename from include/asm-powerpc/mmu-hash32.h rename to arch/powerpc/include/asm/mmu-hash32.h diff --git a/include/asm-powerpc/mmu-hash64.h b/arch/powerpc/include/asm/mmu-hash64.h similarity index 100% rename from include/asm-powerpc/mmu-hash64.h rename to arch/powerpc/include/asm/mmu-hash64.h diff --git a/include/asm-powerpc/mmu.h b/arch/powerpc/include/asm/mmu.h similarity index 100% rename from include/asm-powerpc/mmu.h rename to arch/powerpc/include/asm/mmu.h diff --git a/include/asm-powerpc/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h similarity index 100% rename from include/asm-powerpc/mmu_context.h rename to arch/powerpc/include/asm/mmu_context.h diff --git a/include/asm-powerpc/mmzone.h b/arch/powerpc/include/asm/mmzone.h similarity index 100% rename from include/asm-powerpc/mmzone.h rename to arch/powerpc/include/asm/mmzone.h diff --git a/include/asm-powerpc/module.h b/arch/powerpc/include/asm/module.h similarity index 100% rename from include/asm-powerpc/module.h rename to arch/powerpc/include/asm/module.h diff --git a/include/asm-powerpc/mpc512x.h b/arch/powerpc/include/asm/mpc512x.h similarity index 100% rename from include/asm-powerpc/mpc512x.h rename to arch/powerpc/include/asm/mpc512x.h diff --git a/include/asm-powerpc/mpc52xx.h b/arch/powerpc/include/asm/mpc52xx.h similarity index 100% rename from include/asm-powerpc/mpc52xx.h rename to arch/powerpc/include/asm/mpc52xx.h diff --git a/include/asm-powerpc/mpc52xx_psc.h b/arch/powerpc/include/asm/mpc52xx_psc.h similarity index 100% rename from include/asm-powerpc/mpc52xx_psc.h rename to arch/powerpc/include/asm/mpc52xx_psc.h diff --git a/include/asm-powerpc/mpc6xx.h b/arch/powerpc/include/asm/mpc6xx.h similarity index 100% rename from include/asm-powerpc/mpc6xx.h rename to arch/powerpc/include/asm/mpc6xx.h diff --git a/include/asm-powerpc/mpc8260.h b/arch/powerpc/include/asm/mpc8260.h similarity index 100% rename from include/asm-powerpc/mpc8260.h rename to arch/powerpc/include/asm/mpc8260.h diff --git a/include/asm-powerpc/mpc86xx.h b/arch/powerpc/include/asm/mpc86xx.h similarity index 100% rename from include/asm-powerpc/mpc86xx.h rename to arch/powerpc/include/asm/mpc86xx.h diff --git a/include/asm-powerpc/mpc8xx.h b/arch/powerpc/include/asm/mpc8xx.h similarity index 100% rename from include/asm-powerpc/mpc8xx.h rename to arch/powerpc/include/asm/mpc8xx.h diff --git a/include/asm-powerpc/mpic.h b/arch/powerpc/include/asm/mpic.h similarity index 100% rename from include/asm-powerpc/mpic.h rename to arch/powerpc/include/asm/mpic.h diff --git a/include/asm-powerpc/msgbuf.h b/arch/powerpc/include/asm/msgbuf.h similarity index 100% rename from include/asm-powerpc/msgbuf.h rename to arch/powerpc/include/asm/msgbuf.h diff --git a/include/asm-powerpc/mutex.h b/arch/powerpc/include/asm/mutex.h similarity index 100% rename from include/asm-powerpc/mutex.h rename to arch/powerpc/include/asm/mutex.h diff --git a/include/asm-powerpc/nvram.h b/arch/powerpc/include/asm/nvram.h similarity index 100% rename from include/asm-powerpc/nvram.h rename to arch/powerpc/include/asm/nvram.h diff --git a/include/asm-powerpc/of_device.h b/arch/powerpc/include/asm/of_device.h similarity index 100% rename from include/asm-powerpc/of_device.h rename to arch/powerpc/include/asm/of_device.h diff --git a/include/asm-powerpc/of_platform.h b/arch/powerpc/include/asm/of_platform.h similarity index 100% rename from include/asm-powerpc/of_platform.h rename to arch/powerpc/include/asm/of_platform.h diff --git a/include/asm-powerpc/ohare.h b/arch/powerpc/include/asm/ohare.h similarity index 100% rename from include/asm-powerpc/ohare.h rename to arch/powerpc/include/asm/ohare.h diff --git a/include/asm-powerpc/oprofile_impl.h b/arch/powerpc/include/asm/oprofile_impl.h similarity index 100% rename from include/asm-powerpc/oprofile_impl.h rename to arch/powerpc/include/asm/oprofile_impl.h diff --git a/include/asm-powerpc/pSeries_reconfig.h b/arch/powerpc/include/asm/pSeries_reconfig.h similarity index 100% rename from include/asm-powerpc/pSeries_reconfig.h rename to arch/powerpc/include/asm/pSeries_reconfig.h diff --git a/include/asm-powerpc/paca.h b/arch/powerpc/include/asm/paca.h similarity index 99% rename from include/asm-powerpc/paca.h rename to arch/powerpc/include/asm/paca.h index 7b56444..6493a39 100644 --- a/include/asm-powerpc/paca.h +++ b/arch/powerpc/include/asm/paca.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/paca.h - * * This control block defines the PACA which defines the processor * specific data for each logical processor on the system. * There are some pointers defined that are utilized by PLIC. diff --git a/include/asm-powerpc/page.h b/arch/powerpc/include/asm/page.h similarity index 100% rename from include/asm-powerpc/page.h rename to arch/powerpc/include/asm/page.h diff --git a/include/asm-powerpc/page_32.h b/arch/powerpc/include/asm/page_32.h similarity index 100% rename from include/asm-powerpc/page_32.h rename to arch/powerpc/include/asm/page_32.h diff --git a/include/asm-powerpc/page_64.h b/arch/powerpc/include/asm/page_64.h similarity index 100% rename from include/asm-powerpc/page_64.h rename to arch/powerpc/include/asm/page_64.h diff --git a/include/asm-powerpc/param.h b/arch/powerpc/include/asm/param.h similarity index 100% rename from include/asm-powerpc/param.h rename to arch/powerpc/include/asm/param.h diff --git a/include/asm-powerpc/parport.h b/arch/powerpc/include/asm/parport.h similarity index 100% rename from include/asm-powerpc/parport.h rename to arch/powerpc/include/asm/parport.h diff --git a/include/asm-powerpc/pasemi_dma.h b/arch/powerpc/include/asm/pasemi_dma.h similarity index 100% rename from include/asm-powerpc/pasemi_dma.h rename to arch/powerpc/include/asm/pasemi_dma.h diff --git a/include/asm-powerpc/pci-bridge.h b/arch/powerpc/include/asm/pci-bridge.h similarity index 100% rename from include/asm-powerpc/pci-bridge.h rename to arch/powerpc/include/asm/pci-bridge.h diff --git a/include/asm-powerpc/pci.h b/arch/powerpc/include/asm/pci.h similarity index 100% rename from include/asm-powerpc/pci.h rename to arch/powerpc/include/asm/pci.h diff --git a/include/asm-powerpc/percpu.h b/arch/powerpc/include/asm/percpu.h similarity index 100% rename from include/asm-powerpc/percpu.h rename to arch/powerpc/include/asm/percpu.h diff --git a/include/asm-powerpc/pgalloc-32.h b/arch/powerpc/include/asm/pgalloc-32.h similarity index 100% rename from include/asm-powerpc/pgalloc-32.h rename to arch/powerpc/include/asm/pgalloc-32.h diff --git a/include/asm-powerpc/pgalloc-64.h b/arch/powerpc/include/asm/pgalloc-64.h similarity index 100% rename from include/asm-powerpc/pgalloc-64.h rename to arch/powerpc/include/asm/pgalloc-64.h diff --git a/include/asm-powerpc/pgalloc.h b/arch/powerpc/include/asm/pgalloc.h similarity index 100% rename from include/asm-powerpc/pgalloc.h rename to arch/powerpc/include/asm/pgalloc.h diff --git a/include/asm-powerpc/pgtable-4k.h b/arch/powerpc/include/asm/pgtable-4k.h similarity index 100% rename from include/asm-powerpc/pgtable-4k.h rename to arch/powerpc/include/asm/pgtable-4k.h diff --git a/include/asm-powerpc/pgtable-64k.h b/arch/powerpc/include/asm/pgtable-64k.h similarity index 100% rename from include/asm-powerpc/pgtable-64k.h rename to arch/powerpc/include/asm/pgtable-64k.h diff --git a/include/asm-powerpc/pgtable-ppc32.h b/arch/powerpc/include/asm/pgtable-ppc32.h similarity index 100% rename from include/asm-powerpc/pgtable-ppc32.h rename to arch/powerpc/include/asm/pgtable-ppc32.h diff --git a/include/asm-powerpc/pgtable-ppc64.h b/arch/powerpc/include/asm/pgtable-ppc64.h similarity index 99% rename from include/asm-powerpc/pgtable-ppc64.h rename to arch/powerpc/include/asm/pgtable-ppc64.h index 74c6f38..db0b8f3 100644 --- a/include/asm-powerpc/pgtable-ppc64.h +++ b/arch/powerpc/include/asm/pgtable-ppc64.h @@ -100,7 +100,7 @@ #define _PAGE_WRENABLE (_PAGE_RW | _PAGE_DIRTY) -/* __pgprot defined in asm-powerpc/page.h */ +/* __pgprot defined in arch/powerpc/incliude/asm/page.h */ #define PAGE_NONE __pgprot(_PAGE_PRESENT | _PAGE_ACCESSED) #define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_RW | _PAGE_USER) diff --git a/include/asm-powerpc/pgtable.h b/arch/powerpc/include/asm/pgtable.h similarity index 100% rename from include/asm-powerpc/pgtable.h rename to arch/powerpc/include/asm/pgtable.h diff --git a/include/asm-powerpc/phyp_dump.h b/arch/powerpc/include/asm/phyp_dump.h similarity index 100% rename from include/asm-powerpc/phyp_dump.h rename to arch/powerpc/include/asm/phyp_dump.h diff --git a/include/asm-powerpc/pmac_feature.h b/arch/powerpc/include/asm/pmac_feature.h similarity index 100% rename from include/asm-powerpc/pmac_feature.h rename to arch/powerpc/include/asm/pmac_feature.h diff --git a/include/asm-powerpc/pmac_low_i2c.h b/arch/powerpc/include/asm/pmac_low_i2c.h similarity index 100% rename from include/asm-powerpc/pmac_low_i2c.h rename to arch/powerpc/include/asm/pmac_low_i2c.h diff --git a/include/asm-powerpc/pmac_pfunc.h b/arch/powerpc/include/asm/pmac_pfunc.h similarity index 100% rename from include/asm-powerpc/pmac_pfunc.h rename to arch/powerpc/include/asm/pmac_pfunc.h diff --git a/include/asm-powerpc/pmc.h b/arch/powerpc/include/asm/pmc.h similarity index 100% rename from include/asm-powerpc/pmc.h rename to arch/powerpc/include/asm/pmc.h diff --git a/include/asm-powerpc/pmi.h b/arch/powerpc/include/asm/pmi.h similarity index 100% rename from include/asm-powerpc/pmi.h rename to arch/powerpc/include/asm/pmi.h diff --git a/include/asm-powerpc/poll.h b/arch/powerpc/include/asm/poll.h similarity index 100% rename from include/asm-powerpc/poll.h rename to arch/powerpc/include/asm/poll.h diff --git a/include/asm-powerpc/posix_types.h b/arch/powerpc/include/asm/posix_types.h similarity index 100% rename from include/asm-powerpc/posix_types.h rename to arch/powerpc/include/asm/posix_types.h diff --git a/include/asm-powerpc/ppc-pci.h b/arch/powerpc/include/asm/ppc-pci.h similarity index 100% rename from include/asm-powerpc/ppc-pci.h rename to arch/powerpc/include/asm/ppc-pci.h diff --git a/include/asm-powerpc/ppc4xx.h b/arch/powerpc/include/asm/ppc4xx.h similarity index 100% rename from include/asm-powerpc/ppc4xx.h rename to arch/powerpc/include/asm/ppc4xx.h diff --git a/include/asm-powerpc/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h similarity index 100% rename from include/asm-powerpc/ppc_asm.h rename to arch/powerpc/include/asm/ppc_asm.h diff --git a/include/asm-powerpc/processor.h b/arch/powerpc/include/asm/processor.h similarity index 100% rename from include/asm-powerpc/processor.h rename to arch/powerpc/include/asm/processor.h diff --git a/include/asm-powerpc/prom.h b/arch/powerpc/include/asm/prom.h similarity index 100% rename from include/asm-powerpc/prom.h rename to arch/powerpc/include/asm/prom.h diff --git a/include/asm-powerpc/ps3.h b/arch/powerpc/include/asm/ps3.h similarity index 100% rename from include/asm-powerpc/ps3.h rename to arch/powerpc/include/asm/ps3.h diff --git a/include/asm-powerpc/ps3av.h b/arch/powerpc/include/asm/ps3av.h similarity index 100% rename from include/asm-powerpc/ps3av.h rename to arch/powerpc/include/asm/ps3av.h diff --git a/include/asm-powerpc/ps3fb.h b/arch/powerpc/include/asm/ps3fb.h similarity index 100% rename from include/asm-powerpc/ps3fb.h rename to arch/powerpc/include/asm/ps3fb.h diff --git a/include/asm-powerpc/ps3stor.h b/arch/powerpc/include/asm/ps3stor.h similarity index 100% rename from include/asm-powerpc/ps3stor.h rename to arch/powerpc/include/asm/ps3stor.h diff --git a/include/asm-powerpc/ptrace.h b/arch/powerpc/include/asm/ptrace.h similarity index 100% rename from include/asm-powerpc/ptrace.h rename to arch/powerpc/include/asm/ptrace.h diff --git a/include/asm-powerpc/qe.h b/arch/powerpc/include/asm/qe.h similarity index 100% rename from include/asm-powerpc/qe.h rename to arch/powerpc/include/asm/qe.h diff --git a/include/asm-powerpc/qe_ic.h b/arch/powerpc/include/asm/qe_ic.h similarity index 99% rename from include/asm-powerpc/qe_ic.h rename to arch/powerpc/include/asm/qe_ic.h index a779b2c..56a7745 100644 --- a/include/asm-powerpc/qe_ic.h +++ b/arch/powerpc/include/asm/qe_ic.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/qe_ic.h - * * Copyright (C) 2006 Freescale Semicondutor, Inc. All rights reserved. * * Authors: Shlomi Gridish diff --git a/include/asm-powerpc/reg.h b/arch/powerpc/include/asm/reg.h similarity index 100% rename from include/asm-powerpc/reg.h rename to arch/powerpc/include/asm/reg.h diff --git a/include/asm-powerpc/reg_8xx.h b/arch/powerpc/include/asm/reg_8xx.h similarity index 100% rename from include/asm-powerpc/reg_8xx.h rename to arch/powerpc/include/asm/reg_8xx.h diff --git a/include/asm-powerpc/reg_booke.h b/arch/powerpc/include/asm/reg_booke.h similarity index 100% rename from include/asm-powerpc/reg_booke.h rename to arch/powerpc/include/asm/reg_booke.h diff --git a/include/asm-powerpc/reg_fsl_emb.h b/arch/powerpc/include/asm/reg_fsl_emb.h similarity index 100% rename from include/asm-powerpc/reg_fsl_emb.h rename to arch/powerpc/include/asm/reg_fsl_emb.h diff --git a/include/asm-powerpc/resource.h b/arch/powerpc/include/asm/resource.h similarity index 100% rename from include/asm-powerpc/resource.h rename to arch/powerpc/include/asm/resource.h diff --git a/include/asm-powerpc/rheap.h b/arch/powerpc/include/asm/rheap.h similarity index 100% rename from include/asm-powerpc/rheap.h rename to arch/powerpc/include/asm/rheap.h diff --git a/include/asm-powerpc/rio.h b/arch/powerpc/include/asm/rio.h similarity index 100% rename from include/asm-powerpc/rio.h rename to arch/powerpc/include/asm/rio.h diff --git a/include/asm-powerpc/rtas.h b/arch/powerpc/include/asm/rtas.h similarity index 100% rename from include/asm-powerpc/rtas.h rename to arch/powerpc/include/asm/rtas.h diff --git a/include/asm-powerpc/rtc.h b/arch/powerpc/include/asm/rtc.h similarity index 100% rename from include/asm-powerpc/rtc.h rename to arch/powerpc/include/asm/rtc.h diff --git a/include/asm-powerpc/rwsem.h b/arch/powerpc/include/asm/rwsem.h similarity index 96% rename from include/asm-powerpc/rwsem.h rename to arch/powerpc/include/asm/rwsem.h index a6cc93b..24cd928 100644 --- a/include/asm-powerpc/rwsem.h +++ b/arch/powerpc/include/asm/rwsem.h @@ -8,8 +8,8 @@ #ifdef __KERNEL__ /* - * include/asm-powerpc/rwsem.h: R/W semaphores for PPC using the stuff - * in lib/rwsem.c. Adapted largely from include/asm-i386/rwsem.h + * R/W semaphores for PPC using the stuff in lib/rwsem.c. + * Adapted largely from include/asm-i386/rwsem.h * by Paul Mackerras . */ diff --git a/include/asm-powerpc/scatterlist.h b/arch/powerpc/include/asm/scatterlist.h similarity index 100% rename from include/asm-powerpc/scatterlist.h rename to arch/powerpc/include/asm/scatterlist.h diff --git a/include/asm-powerpc/seccomp.h b/arch/powerpc/include/asm/seccomp.h similarity index 100% rename from include/asm-powerpc/seccomp.h rename to arch/powerpc/include/asm/seccomp.h diff --git a/include/asm-powerpc/sections.h b/arch/powerpc/include/asm/sections.h similarity index 100% rename from include/asm-powerpc/sections.h rename to arch/powerpc/include/asm/sections.h diff --git a/include/asm-powerpc/sembuf.h b/arch/powerpc/include/asm/sembuf.h similarity index 100% rename from include/asm-powerpc/sembuf.h rename to arch/powerpc/include/asm/sembuf.h diff --git a/include/asm-powerpc/serial.h b/arch/powerpc/include/asm/serial.h similarity index 100% rename from include/asm-powerpc/serial.h rename to arch/powerpc/include/asm/serial.h diff --git a/include/asm-powerpc/setjmp.h b/arch/powerpc/include/asm/setjmp.h similarity index 100% rename from include/asm-powerpc/setjmp.h rename to arch/powerpc/include/asm/setjmp.h diff --git a/include/asm-powerpc/setup.h b/arch/powerpc/include/asm/setup.h similarity index 100% rename from include/asm-powerpc/setup.h rename to arch/powerpc/include/asm/setup.h diff --git a/include/asm-powerpc/shmbuf.h b/arch/powerpc/include/asm/shmbuf.h similarity index 100% rename from include/asm-powerpc/shmbuf.h rename to arch/powerpc/include/asm/shmbuf.h diff --git a/include/asm-powerpc/shmparam.h b/arch/powerpc/include/asm/shmparam.h similarity index 100% rename from include/asm-powerpc/shmparam.h rename to arch/powerpc/include/asm/shmparam.h diff --git a/include/asm-powerpc/sigcontext.h b/arch/powerpc/include/asm/sigcontext.h similarity index 100% rename from include/asm-powerpc/sigcontext.h rename to arch/powerpc/include/asm/sigcontext.h diff --git a/include/asm-powerpc/siginfo.h b/arch/powerpc/include/asm/siginfo.h similarity index 100% rename from include/asm-powerpc/siginfo.h rename to arch/powerpc/include/asm/siginfo.h diff --git a/include/asm-powerpc/signal.h b/arch/powerpc/include/asm/signal.h similarity index 100% rename from include/asm-powerpc/signal.h rename to arch/powerpc/include/asm/signal.h diff --git a/include/asm-powerpc/smp.h b/arch/powerpc/include/asm/smp.h similarity index 100% rename from include/asm-powerpc/smp.h rename to arch/powerpc/include/asm/smp.h diff --git a/include/asm-powerpc/smu.h b/arch/powerpc/include/asm/smu.h similarity index 100% rename from include/asm-powerpc/smu.h rename to arch/powerpc/include/asm/smu.h diff --git a/include/asm-powerpc/socket.h b/arch/powerpc/include/asm/socket.h similarity index 100% rename from include/asm-powerpc/socket.h rename to arch/powerpc/include/asm/socket.h diff --git a/include/asm-powerpc/sockios.h b/arch/powerpc/include/asm/sockios.h similarity index 100% rename from include/asm-powerpc/sockios.h rename to arch/powerpc/include/asm/sockios.h diff --git a/include/asm-powerpc/sparsemem.h b/arch/powerpc/include/asm/sparsemem.h similarity index 100% rename from include/asm-powerpc/sparsemem.h rename to arch/powerpc/include/asm/sparsemem.h diff --git a/include/asm-powerpc/spinlock.h b/arch/powerpc/include/asm/spinlock.h similarity index 100% rename from include/asm-powerpc/spinlock.h rename to arch/powerpc/include/asm/spinlock.h diff --git a/include/asm-powerpc/spinlock_types.h b/arch/powerpc/include/asm/spinlock_types.h similarity index 100% rename from include/asm-powerpc/spinlock_types.h rename to arch/powerpc/include/asm/spinlock_types.h diff --git a/include/asm-powerpc/spu.h b/arch/powerpc/include/asm/spu.h similarity index 100% rename from include/asm-powerpc/spu.h rename to arch/powerpc/include/asm/spu.h diff --git a/include/asm-powerpc/spu_csa.h b/arch/powerpc/include/asm/spu_csa.h similarity index 100% rename from include/asm-powerpc/spu_csa.h rename to arch/powerpc/include/asm/spu_csa.h diff --git a/include/asm-powerpc/spu_info.h b/arch/powerpc/include/asm/spu_info.h similarity index 100% rename from include/asm-powerpc/spu_info.h rename to arch/powerpc/include/asm/spu_info.h diff --git a/include/asm-powerpc/spu_priv1.h b/arch/powerpc/include/asm/spu_priv1.h similarity index 100% rename from include/asm-powerpc/spu_priv1.h rename to arch/powerpc/include/asm/spu_priv1.h diff --git a/include/asm-powerpc/sstep.h b/arch/powerpc/include/asm/sstep.h similarity index 100% rename from include/asm-powerpc/sstep.h rename to arch/powerpc/include/asm/sstep.h diff --git a/include/asm-powerpc/stat.h b/arch/powerpc/include/asm/stat.h similarity index 100% rename from include/asm-powerpc/stat.h rename to arch/powerpc/include/asm/stat.h diff --git a/include/asm-powerpc/statfs.h b/arch/powerpc/include/asm/statfs.h similarity index 100% rename from include/asm-powerpc/statfs.h rename to arch/powerpc/include/asm/statfs.h diff --git a/include/asm-powerpc/string.h b/arch/powerpc/include/asm/string.h similarity index 100% rename from include/asm-powerpc/string.h rename to arch/powerpc/include/asm/string.h diff --git a/include/asm-powerpc/suspend.h b/arch/powerpc/include/asm/suspend.h similarity index 100% rename from include/asm-powerpc/suspend.h rename to arch/powerpc/include/asm/suspend.h diff --git a/include/asm-powerpc/synch.h b/arch/powerpc/include/asm/synch.h similarity index 100% rename from include/asm-powerpc/synch.h rename to arch/powerpc/include/asm/synch.h diff --git a/include/asm-powerpc/syscall.h b/arch/powerpc/include/asm/syscall.h similarity index 100% rename from include/asm-powerpc/syscall.h rename to arch/powerpc/include/asm/syscall.h diff --git a/include/asm-powerpc/syscalls.h b/arch/powerpc/include/asm/syscalls.h similarity index 100% rename from include/asm-powerpc/syscalls.h rename to arch/powerpc/include/asm/syscalls.h diff --git a/include/asm-powerpc/systbl.h b/arch/powerpc/include/asm/systbl.h similarity index 100% rename from include/asm-powerpc/systbl.h rename to arch/powerpc/include/asm/systbl.h diff --git a/include/asm-powerpc/system.h b/arch/powerpc/include/asm/system.h similarity index 100% rename from include/asm-powerpc/system.h rename to arch/powerpc/include/asm/system.h diff --git a/include/asm-powerpc/tce.h b/arch/powerpc/include/asm/tce.h similarity index 100% rename from include/asm-powerpc/tce.h rename to arch/powerpc/include/asm/tce.h diff --git a/include/asm-powerpc/termbits.h b/arch/powerpc/include/asm/termbits.h similarity index 100% rename from include/asm-powerpc/termbits.h rename to arch/powerpc/include/asm/termbits.h diff --git a/include/asm-powerpc/termios.h b/arch/powerpc/include/asm/termios.h similarity index 100% rename from include/asm-powerpc/termios.h rename to arch/powerpc/include/asm/termios.h diff --git a/include/asm-powerpc/thread_info.h b/arch/powerpc/include/asm/thread_info.h similarity index 100% rename from include/asm-powerpc/thread_info.h rename to arch/powerpc/include/asm/thread_info.h diff --git a/include/asm-powerpc/time.h b/arch/powerpc/include/asm/time.h similarity index 100% rename from include/asm-powerpc/time.h rename to arch/powerpc/include/asm/time.h diff --git a/include/asm-powerpc/timex.h b/arch/powerpc/include/asm/timex.h similarity index 100% rename from include/asm-powerpc/timex.h rename to arch/powerpc/include/asm/timex.h diff --git a/include/asm-powerpc/tlb.h b/arch/powerpc/include/asm/tlb.h similarity index 100% rename from include/asm-powerpc/tlb.h rename to arch/powerpc/include/asm/tlb.h diff --git a/include/asm-powerpc/tlbflush.h b/arch/powerpc/include/asm/tlbflush.h similarity index 100% rename from include/asm-powerpc/tlbflush.h rename to arch/powerpc/include/asm/tlbflush.h diff --git a/include/asm-powerpc/topology.h b/arch/powerpc/include/asm/topology.h similarity index 100% rename from include/asm-powerpc/topology.h rename to arch/powerpc/include/asm/topology.h diff --git a/include/asm-powerpc/tsi108.h b/arch/powerpc/include/asm/tsi108.h similarity index 100% rename from include/asm-powerpc/tsi108.h rename to arch/powerpc/include/asm/tsi108.h diff --git a/include/asm-powerpc/tsi108_irq.h b/arch/powerpc/include/asm/tsi108_irq.h similarity index 100% rename from include/asm-powerpc/tsi108_irq.h rename to arch/powerpc/include/asm/tsi108_irq.h diff --git a/include/asm-powerpc/tsi108_pci.h b/arch/powerpc/include/asm/tsi108_pci.h similarity index 100% rename from include/asm-powerpc/tsi108_pci.h rename to arch/powerpc/include/asm/tsi108_pci.h diff --git a/include/asm-powerpc/types.h b/arch/powerpc/include/asm/types.h similarity index 100% rename from include/asm-powerpc/types.h rename to arch/powerpc/include/asm/types.h diff --git a/include/asm-powerpc/uaccess.h b/arch/powerpc/include/asm/uaccess.h similarity index 100% rename from include/asm-powerpc/uaccess.h rename to arch/powerpc/include/asm/uaccess.h diff --git a/include/asm-powerpc/ucc.h b/arch/powerpc/include/asm/ucc.h similarity index 100% rename from include/asm-powerpc/ucc.h rename to arch/powerpc/include/asm/ucc.h diff --git a/include/asm-powerpc/ucc_fast.h b/arch/powerpc/include/asm/ucc_fast.h similarity index 99% rename from include/asm-powerpc/ucc_fast.h rename to arch/powerpc/include/asm/ucc_fast.h index fce16ab..839aab8 100644 --- a/include/asm-powerpc/ucc_fast.h +++ b/arch/powerpc/include/asm/ucc_fast.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/ucc_fast.h - * * Internal header file for UCC FAST unit routines. * * Copyright (C) 2006 Freescale Semicondutor, Inc. All rights reserved. diff --git a/include/asm-powerpc/ucc_slow.h b/arch/powerpc/include/asm/ucc_slow.h similarity index 100% rename from include/asm-powerpc/ucc_slow.h rename to arch/powerpc/include/asm/ucc_slow.h diff --git a/include/asm-powerpc/ucontext.h b/arch/powerpc/include/asm/ucontext.h similarity index 100% rename from include/asm-powerpc/ucontext.h rename to arch/powerpc/include/asm/ucontext.h diff --git a/include/asm-powerpc/udbg.h b/arch/powerpc/include/asm/udbg.h similarity index 100% rename from include/asm-powerpc/udbg.h rename to arch/powerpc/include/asm/udbg.h diff --git a/include/asm-powerpc/uic.h b/arch/powerpc/include/asm/uic.h similarity index 95% rename from include/asm-powerpc/uic.h rename to arch/powerpc/include/asm/uic.h index 970eb7e..597edfc 100644 --- a/include/asm-powerpc/uic.h +++ b/arch/powerpc/include/asm/uic.h @@ -1,6 +1,4 @@ /* - * include/asm-powerpc/uic.h - * * IBM PPC4xx UIC external definitions and structure. * * Maintainer: David Gibson diff --git a/include/asm-powerpc/unaligned.h b/arch/powerpc/include/asm/unaligned.h similarity index 100% rename from include/asm-powerpc/unaligned.h rename to arch/powerpc/include/asm/unaligned.h diff --git a/include/asm-powerpc/uninorth.h b/arch/powerpc/include/asm/uninorth.h similarity index 100% rename from include/asm-powerpc/uninorth.h rename to arch/powerpc/include/asm/uninorth.h diff --git a/include/asm-powerpc/unistd.h b/arch/powerpc/include/asm/unistd.h similarity index 100% rename from include/asm-powerpc/unistd.h rename to arch/powerpc/include/asm/unistd.h diff --git a/include/asm-powerpc/user.h b/arch/powerpc/include/asm/user.h similarity index 100% rename from include/asm-powerpc/user.h rename to arch/powerpc/include/asm/user.h diff --git a/include/asm-powerpc/vdso.h b/arch/powerpc/include/asm/vdso.h similarity index 100% rename from include/asm-powerpc/vdso.h rename to arch/powerpc/include/asm/vdso.h diff --git a/include/asm-powerpc/vdso_datapage.h b/arch/powerpc/include/asm/vdso_datapage.h similarity index 100% rename from include/asm-powerpc/vdso_datapage.h rename to arch/powerpc/include/asm/vdso_datapage.h diff --git a/include/asm-powerpc/vga.h b/arch/powerpc/include/asm/vga.h similarity index 100% rename from include/asm-powerpc/vga.h rename to arch/powerpc/include/asm/vga.h diff --git a/include/asm-powerpc/vio.h b/arch/powerpc/include/asm/vio.h similarity index 100% rename from include/asm-powerpc/vio.h rename to arch/powerpc/include/asm/vio.h diff --git a/include/asm-powerpc/xilinx_intc.h b/arch/powerpc/include/asm/xilinx_intc.h similarity index 100% rename from include/asm-powerpc/xilinx_intc.h rename to arch/powerpc/include/asm/xilinx_intc.h diff --git a/include/asm-powerpc/xmon.h b/arch/powerpc/include/asm/xmon.h similarity index 100% rename from include/asm-powerpc/xmon.h rename to arch/powerpc/include/asm/xmon.h diff --git a/include/asm-powerpc/xor.h b/arch/powerpc/include/asm/xor.h similarity index 100% rename from include/asm-powerpc/xor.h rename to arch/powerpc/include/asm/xor.h diff --git a/arch/powerpc/mm/tlb_64.c b/arch/powerpc/mm/tlb_64.c index 409fcc7..be7dd42 100644 --- a/arch/powerpc/mm/tlb_64.c +++ b/arch/powerpc/mm/tlb_64.c @@ -34,7 +34,7 @@ DEFINE_PER_CPU(struct ppc64_tlb_batch, ppc64_tlb_batch); /* This is declared as we are using the more or less generic - * include/asm-powerpc/tlb.h file -- tgall + * arch/powerpc/include/asm/tlb.h file -- tgall */ DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); static DEFINE_PER_CPU(struct pte_freelist_batch *, pte_freelist_cur); diff --git a/arch/powerpc/platforms/86xx/mpc86xx_smp.c b/arch/powerpc/platforms/86xx/mpc86xx_smp.c index 835f2dc..014e26c 100644 --- a/arch/powerpc/platforms/86xx/mpc86xx_smp.c +++ b/arch/powerpc/platforms/86xx/mpc86xx_smp.c @@ -19,7 +19,7 @@ #include #include #include -#include +#include #include #include diff --git a/drivers/char/hvc_console.h b/drivers/char/hvc_console.h index d9ce109..9790201 100644 --- a/drivers/char/hvc_console.h +++ b/drivers/char/hvc_console.h @@ -6,7 +6,7 @@ * Ryan S. Arnold * * hvc_console header information: - * moved here from include/asm-powerpc/hvconsole.h + * moved here from arch/powerpc/include/asm/hvconsole.h * and drivers/char/hvc_console.c * * This program is free software; you can redistribute it and/or modify diff --git a/drivers/char/hvcs.c b/drivers/char/hvcs.c index 786d518..473d9b1 100644 --- a/drivers/char/hvcs.c +++ b/drivers/char/hvcs.c @@ -114,7 +114,7 @@ * the hvcs_final_close() function in order to get it out of the spinlock. * Rearranged hvcs_close(). Cleaned up some printks and did some housekeeping * on the changelog. Removed local CLC_LENGTH and used HVCS_CLC_LENGTH from - * include/asm-powerpc/hvcserver.h + * arch/powerepc/include/asm/hvcserver.h * * 1.3.2 -> 1.3.3 Replaced yield() in hvcs_close() with tty_wait_until_sent() to * prevent possible lockup with realtime scheduling as similarily pointed out by diff --git a/drivers/infiniband/hw/ehca/ehca_reqs.c b/drivers/infiniband/hw/ehca/ehca_reqs.c index dd9bc68..898c8b5 100644 --- a/drivers/infiniband/hw/ehca/ehca_reqs.c +++ b/drivers/infiniband/hw/ehca/ehca_reqs.c @@ -42,7 +42,7 @@ */ -#include +#include #include "ehca_classes.h" #include "ehca_tools.h" #include "ehca_qes.h" -- 1.5.6.3 -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/